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The AISI Colloquium 



The Alberta Initiative for School Improvement (AISI) is a province-wide partnership 
program whose goal is to improve student learning and performance by fostering 
initiatives that reflect the unique needs and circumstances of each school authority. It is 
currently ending its third cycle and ninth year of implementation. 

The Colloquium on Large Scale Improvement: Implications for AISI took place in 
Edmonton, October 20-22, 2008. The purposes of the colloquium were to share 
information about AISI from multiple perspectives and to set a strategic direction for the 
future of the program. The Honourable Dave Hancock, Minister of Education, 
announced Cycle 4 of AISI when he opened the colloquium. 

The colloquium brought together three perspectives on the program - that of AISI partners, 
school authorities, and experts in areas of strategic importance to AISI. Representatives from 
these groups made presentations, participated in the deliberations, and wrote chapters for this 
report. 

The AISI partners consist of representatives from the following organizations: 

• Alberta Education 

• Alberta School Boards Association (ASBA) 

• Alberta School Councils’ Association (ASCA) 

• Alberta Teachers’ Association (AT A) 

• Association of School Business Officials of Alberta (ASBOA) 

• College of Alberta School Superintendents (CASS) 

• University Faculties of Education (Alberta, Calgary, Lethbridge) 

Six school authorities were invited to present a field perspective on AISI. They represent 
different types of school authority and all regions of the province. 

• Peace Wapiti School Division 

• Edmonton Public Schools 

• Pembina Hills Regional Division 

• Wolf Creek School Division 

• Calgary Catholic School District 

• Prairie Rose School division 

Five experts were invited to share their expertise and experience with the group. Areas of 
particular interest to AISI as it begins its second decade are evidence, change, policy, and 
complexity thinking. 

• Evidence - Robert Crocker, Atlantic Evaluation and Research Consultants 

• Change - Andy Hargreaves, Boston College 

• Policy - Pasi Sahlberg, European Training Foundation 

• Complexity Thinking - Brent Davis and Dennis Sumara, University of British Columbia 



iii 
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AISI was developed in 1999 when the partners developed its framework and administrative 
requirements 1 . The AISI Colloquium is the latest in a series of strategic events in which 
partners have identified opportunities, challenges, and implications. 

• In 2000, AISI partners participated in a symposium 2 in which they identified the 
opportunities and challenges they foresaw in this new initiative. 

• In 2005, AISI partners spent a strategic planning day 3 to develop a shared understanding 
of the key components of the AISI framework and issues related to the implementation of 
AISI Cycle 3. The summary of the deliberations was used to inform Cycle 3. 

• In 2007, another symposium 4 brought together four partner organizations to identify 
implications of AISI for professional practice. 

This 2008 AISI Colloquium differed from previous symposia in that it extended both the 
time (three days) and scope (more and varied participants) to reflect on AISI successes 
and propose enhancements for its next cycle. AISI coordinators and superintendents 
from representative school authorities joined the partners and external experts in sharing 
perspectives and recommending suggestions for the enhancement of AISI. 

Following our normal practice, the products from the colloquium have been posted on the 
AISI website. They follow a multimedia approach. In addition to this report, the Power 
Points and videotaped presentations can be found at 
http://education.alberta.ca/admin/aisi/colloquium.aspx 



1 This resulted in the Framework for the Alberta Initiative for School Improvement (AISI Education Partners 
Steering Committee, 1999) and AISI administrative handbook (AISI Education Partners Working, 1999). 

2 Alberta Initiative for School Improvement: Opportunities and challenges. (2000, May). Symposium at 
the annual meeting of the Canadian Society for the Study of Education, Edmonton, AB. 

3 AISI Education Partners Steering Committee strategic planning day. Meeting summary report. (2005, 
December). 

4 Implications of a large-scale improvement initiative for professional practice. (2007, April). Symposium 
at the annual meeting of the American Educational Research Association, Chicago, IL. 
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Chapter 1 

AISI: From Inception to Transformation 

Nelly McEwen and Dianna Millard 
Alberta Education 



The Alberta Initiative for School Improvement (AISI) has been in effect for almost a decade. 
Over this period of time it has matured and flourished. The end of Cycle 3 provides an 
opportunity to reflect on its success, examine its ongoing challenges, and consider future 
directions. 

This chapter provides the provincial perspective on AISI, from its inception to its proposed 
transformation. It includes a synopsis of school improvement and a brief description of the 
AISI program; evidence of the impact of AISI on student learning; and other areas of interest 
to the colloquium - change, policy, and complexity. The chapter ends with the pillars of 
transformation and a conclusion. 



School Improvement 

School improvement focuses on improving both the quality and equity of student learning by 
fostering enhanced strategies at the school, district, and provincial/state levels. Areas that 
promote school improvement include leadership, instructional practice, school climate, 
assessment and accountability, building capacity through professional development, student 
and parent engagement, and sustainability. 

How do large-scale improvement initiatives renew themselves so that their intended 
purposes are met, yet accommodate changing circumstances? Accelerating changes in all 
areas of life - demographic, social, economic, and technological - have an important effect 
on education. Incorporating new knowledge and emerging technologies into the effective 
and efficient operation of schools is imperative. Schools play several roles including 
development of learning, socialization, and community centres for agencies working 
together. Schools must also embrace ways to facilitate student learning “outside the 
school” through technology and partnerships. 

Improving student learning and schools has been on the research and policy agendas for 
many decades. There is an extensive body of research on ways to improve educational 
performance 5 . The following quotation summarizes the impact of this literature. 



5 See for example: Fraser, Walberg, Welch, & Hattie, 1987; Hattie, 1992; Wang, Haertel, & Walberg, 1993; 
Creemers, 1994; Hattie, Biggs, & Purdie, 1996; Scheerens & Bosker, 1997; Wright, Horn, & Sanders, 1997; 
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The cumulative research of the last 40 years provides some clear guidance about the 
characteristics of effective schools and effective teaching. . . . when the research 
undertaken during the last four decades is considered as a set, there is ample evidence 
that schools can and do make a powerful difference in the academic achievement of 
students. (Marzano, 2000, pp. 1-2) 

The research draws on the traditions of school effectiveness and school improvement. 
School effectiveness is more directed to finding out what works in education and why. 
School improvement is practice and policy oriented and intended to change education in 
the desired direction. In the orientation on outcomes, input, processes, and context in 
education, both approaches have much in common. Today the two traditions are usually 
merged (Creemers, 2002, p. 343). Combining the two perspectives has led to using the 
school improvement vehicle and the school effectiveness knowledge base to enlarge our 
understanding of how schools operate and possibilities for improving them. By 
combining elements of both traditions, mixed methods rather than either quantitative or 
qualitative ones are used for description and explanation (Hopkins, 2001, p. 57). 



The Alberta Initiative for School Improvement 

AISI is a bold approach to improving student learning by encouraging teachers, parents, 
and the community to work collaboratively to introduce innovative projects that address 
local needs. Initiated in 1999, AISI provides targeted funding to school authorities to 
improve student learning and performance. In this province-wide school improvement 
program, individual school authorities decide: 

1. Which areas of student learning and performance need attention. 

2. How to go about improving these areas (new teaching strategies, student support, etc.). 

3. How to provide evidence that improvement has taken place (measuring student 
performance). 

The goal of this program is to improve student learning and performance by fostering 
initiatives that reflect the unique needs and circumstances of each school authority. 

The transition from Cycle 3 to Cycle 4 is an opportune time to take stock of what has been 
accomplished and determine future directions for AISI. Three cycles - nine years of 
implementation - provide a wealth of information on what works well, what needs 
adjustment, and what opportunities exist for this unique collaborative partnership in 
Alberta. The AISI Colloquium provides an important vehicle for reflecting on the past and 
aspiring to a preferred future for the program by engaging partners, representative school 
authorities, and external experts in the deliberations. 



Marzano, 1998, 2000; Hedges, 2000; Scheerens, 2000; Teddlie & Reynolds, 2000; Hopkins, 2001; Marzano, 
Pickering, & Pollock, 2001; Molnar, 2002). 
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Inception 



AISI was developed through a collaborative partnership 6 in 1999 and first implemented in all 
Alberta school authorities in the 2000/2001 school year. AISI provides funding for every 
school authority in the province to establish its own improvement project(s) to address local 
student needs and circumstances. Initially allocated for three years from 2000 to 2003, 
funding was extended for a second three-year cycle from 2003 to 2006. Currently in its ninth 
year of implementation, more than $560 million have been invested in this initiative. 

The AISI approach to improving student learning is through partnerships and collaboration in 
a culture of inquiry, collaboration, and continuous improvement. AISI encourages creativity 
and innovation in enhancing strategies to improve student learning based upon local needs. 
All school authorities participate in AISI. The provincial government funds all projects that 
meet criteria 7 , provides a number of supports to project teams, approves annual project 
reports, and analyzes and reports overall provincial results. 

Improving Student Learning: Provincial Report for Cycle 1 (Alberta Learning AISI, 2004) 
documents the processes and outcomes of the AISI program from its inception in 1999 
through its first cycle of implementation from 2000 to 2003. The results indicate that AISI 
had a profound impact on the culture of schools in Alberta. Among the impacts were 
improved student learning, development of a culture of continuous improvement, a 
renewed focus on teaching and learning, better decision making based on evidence, job- 
embedded professional development, and shared and distributed leadership. 



Planning and Reporting 

All school authorities are eligible to receive AISI funds. In order to access them, they 
must submit an AISI Project Proposal. Once the proposal meets all criteria, it is 
approved and becomes the project plan. School authorities must file AISI Project Annual 
Reports for all projects. Once these reports have been reviewed and meet all criteria, 
they are posted on the AISI website. 



6 Alberta Education, Alberta School Boards Association (ASBA), Alberta School Councils’ Association 
(ASCA), Alberta Teachers’ Association (ATA), Association of School Business Officials of Alberta 
(ASBOA), and College of Alberta School Superintendents (CASS). University Faculties of Education 
(Alberta, Calgary, Lethbridge) were invited to join the partnership in spring 2000. 

7 

AISI criteria for project approval include a project description, school community involvement, support of 
implementers, literature and research, improvement goal(s) aligned with strategies and measures, outcome 
measures, baseline(s) and improvement targets, key strategies and processes, evaluation process, integration and 
sustainability, knowledge dissemination and sharing, ongoing administrative support, staffing requirement, 
budget projections, project expense percentages, certification by project coordinator, and certification by the 
superintendent. All requirements for project planning and annual reporting are found in the AISI Handbook for 
Cycle 3 (2006-2009) (AISI Education Partners Working Group, 2006). 
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Provincial reports combine projects to give a provincial picture. Combining results 
across projects is a way of introducing both “replication” and “differentiation” into the 
design. That is, like projects can be clustered to determine if they give consistent results 
and unlike projects can be differentiated to determine if some kinds of initiatives work 
better than others. 

Improving Student Learning: Provincial Report for Cycle 2 (Alberta Education AISI, 
2008) documents the processes and outcomes of the AISI program during its second 
cycle. It draws on a wide range of data from 2003 to 2006 to present findings related to 
implementation support and outcomes. The report compares results from Cycle 1 and 
Cycle 2 and presents an agenda for action based on the findings, and draws conclusions 
and implications. Cycle 2 continued to demonstrate a positive impact on student learning 
and teacher, student, and parent satisfaction. 



Evidence 

How do we know that AISI is having a positive impact on student learning? Multiple sources 
of evidence indicate that Cycles 1 and 2 of AISI have had a positive impact on education in 
Alberta (AISI Project Reports [2001 to 2008]; Alberta Learning AISI, 2004; Parsons, 

McRae, & Taylor, 2006; McEwen, 2007; Alberta Education AISI, 2008). 



Approaches and Data Sources 

In order to demonstrate the effectiveness of AISI, one can ask the following questions. 

Has there been an effect and in what direction is the effect? Has the effect been “caused” 
by the intervention of interest? The use of effect sizes represents a sophisticated approach 
to examining average effects. Effect size essentially measures whether or not an effect is 
clinically important rather than whether it is statistically significant. Indeed, the concept of 
effect size is gradually replacing that of statistical significance as a way of looking at 
research findings and especially as a way of synthesizing results. In AISI, effect sizes are 
computed as improvements over baselines. 

In the AISI program, school authorities are responsible for collecting quantitative and 
qualitative data, and analyzing and reporting results for each project. Reporting 
requirements include descriptive results on prior performance (baseline), annual targets, 
and annual results. School authorities are advised to use multiple methods and data 
sources to provide evidence of the success of their projects. School authorities must 
interpret annual results and propose improvement strategies for subsequent years. AISI 
projects are expected to have an appropriate balance of quantitative and qualitative 
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measures 8 . AISI Project Annual Reports also require project teams to interpret findings 
in four major areas: student outcomes, effective practices, integration and sustainability, 
and a summary. 

In the 2008 Cycle 2 provincial report, measures were grouped into four categories: 

• Provincial Tests - provincial achievement tests (grades 3, 6, and 9) and diploma 
exams (grade 12) 

• Local Assessments - standardized tests and locally determined student achievement 
measures 

• Student and Parent Surveys - Parent satisfaction and student satisfaction, behavior, and 
attitudes 

• Teacher Surveys - Teacher growth (knowledge, skills, and attitudes) and satisfaction 
results 

Combined effect sizes for the various project categories were calculated using a program 
for research synthesis (Borenstein & Rothstein, 1999). During Cycle 2 qualitative data 
were analyzed to summarize types of information provided using QDA Miner (Peladeau, 
2004). 



Change Over Time 

What overall effect did the second cycle of AISI have on student learning (measured by 
provincial tests and local assessments), its intended beneficiaries (students and parents) and its 
participants (teachers)? Average annual effects are based on improvement over a three-year 
average baseline, that is, Cycle 1 results are based on improvement over the average of 1998 to 
2000 and Cycle 2 on a baseline of the average of 2001 to 2003, the first three years of AISI 
implementation. Thus the results represent continuous improvement over two cycles or six years 
of AISI implementation. 

Although projects in Cycle 1 and 2 were different, the AISI goal, principles, criteria, and 
procedures remained constant. Furthermore, the school districts are the same and almost all 
schools in the province participated in AISI. Taking the average of the effects over the two 
cycles, it is possible to estimate average annual benefits over six years. This procedure 
resulted in effect sizes of 0.45 for teacher growth and satisfaction, 0.27 for student and parent 
satisfaction, 0.33 for locally determined student achievement, and 0.11 for student 
achievement measured by provincial tests. These average annual effects can be thought of as 
the return on investment of AISI in improving education in the province for the benefit of its 
students and teachers. Figure 1.1 presents the results. 



8 Most project teams that chose quantitative measures used provincial achievement tests and diploma 
examinations as indicators of success. In addition to provincial tests and exams, school authorities used 
more than 40 different commercially available standardized assessment instruments (e.g., Canadian Tests 
of Basic Skills, Gates-MacGinitie Reading Tests, Schonell Tests, and Brigance Tests) to measure student 
learning. For projects that included affective and behavioral goals, project teams had to develop or adapt 
local measures that required analysis and interpretation in order to report results. 
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Teacher Surveys 

Student & Parent 
Surveys 

Local Assessments 
Provincial Tests 

0.00 0.10 0.20 0.30 0.40 0.50 

Average Annual Effect Size Over Six Years 

Figure 1.1. Average Annual Effects ofAISI Cycles 1 and 2 9 




Participants ’ Perceptions 

The AISI Project Final Report (APFR) requires project teams to provide information in 
four major areas: student outcomes, effective practices, integration and sustainability, 
and a summary. Factors that contributed to project success included time and resources, 
professional development, teacher collaboration and sharing, regular assessment, and 
leadership/staff development. Ongoing challenges included gathering and using data 
(including finding appropriate measures of student performance), professional 
development, time away from the classroom, finding and using substitute teachers, and 
administrator support. Unanticipated results included enthusiasm, support, interest, time, 
assessment, and leadership. Project teams celebrated success through recognition and 
awards, conferences, media coverage, and presentations. These perceptions are 
elaborated and illustrated by quotations in the Cycle 2 provincial report. 



Research Reviews 

In addition to the provincial report, independent research reviews analyzed successful 
AISI projects in order to identify promising practices in selected areas 10 . The intent of 
these reviews is to inform educators on ways to develop and enhance these areas. Each 
independent review uses three sources of data: annual reports from 20 to 35 successful 



9 Source: Alberta Education AISI (2008, p. 76). 

10 Topics include: literacy and language arts; mathematics and numeracy; differentiated instruction; 
character, school climate and student leadership; high school completion; collaborative professional 
development; and technology. In 2009, leadership and sustainability, First Nations, Metis, and Inuit, and 
parent and community engagement will be completed. 
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projects from Cycles 1 and 2; findings from a focus group of representatives from 
selected schools and districts drawn from the above samples; and findings from telephone 
interviews with additional schools or districts from the above samples. 

Reviewers found characteristics that were common to all of the projects analyzed. 
Specifically, they found that successful AISI projects supported: regularly scheduled, 
job-embedded time for teacher collaboration; a common culture of action research and 
shared inquiry; significant leadership support at school and district levels; empowerment 
of participants and ongoing professional growth of teachers; and a clear and shared focus 
on the goal of student learning (Alberta Education AISI, 2008, pp. 54-55). 



Change 

How has AISI evolved over time? Figure 1.2 11 portrays the evolution over the first three 
cycles. Cycle 1 established a foundation of trust between government and education 
stakeholders, and created a model for collaboration that has been used in other government 
initiatives. It also established accountability measures and criteria to provide evidence that 
the initiative is working and set the stage for continuous improvement. Cycle 2 consolidated 
emerging knowledge and synthesized what works, built on the enthusiasm and commitment of 
the first cycle, and expanded AISI’s sphere of influence to more teachers and students in 
Alberta. Cycle 2 can be characterized by integration and sustainability of effective practices 
such as promising instructional interventions, professional development, and administration. 
Cycle 3 continues to build on the accomplishments of the first two cycles; it is characterized 
by collaborative inquiry, a process of systematic investigation, emphasizing innovation and 
research, extending what has been learned based on in-depth analysis of project outcomes, 
enhancing professional practice, and focusing professional development on curricular areas. 



^proving 
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Cycle 2: 2003-2006 



Cycle 1 : 2000-2003 Integration & Sustainability 
Foundation 



Cycle 3: 2006-2009 
Collaborative Inquiry 

► 
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Leading & Learning 

Adaptation 

Adoption 



Figure 1.2. The Evolution of AISI over Time 



11 An alternative portrayal exists in the AISI handbook for Cycle 3 (2006, p. 8). The figure above includes a 
temporal dimension whereas the elliptical one in the handbook expands the areas of each cycle. 
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Remaining true to an initiative’s goal, principles, and processes, yet letting it evolve in 
desired ways requires ingenuity. Emerging knowledge and technologies have much to 
contribute. For example, the internet has virtually changed the way people identify 
contacts and resources. More and more material is being posted on the web for ready 
public access. And the increasing sophistication of communications technology is 
making it easier to overcome barriers of time and space to work with others regardless of 
where they are. 

Just as an initiative evolves over time, so too do organizations, groups, and individuals 
participating in it. Participants adapt to changes in the environment. Change involves a 
commitment to lifelong learning through ongoing capacity building. Activities that foster 
renewal usually require funding. Typically improvement or reform initiatives are set for 
three years. This time frame permits the development and/or adoption of new strategies 
aligned with measurement tools; a full cycle of implementation with opportunity for 
refinement in the final year; and collecting evidence of success and analyzing, interpreting, 
and reporting of impact and proposed changes for integrating and sustaining what was 
learned. Generally people are willing to commit for this period of time. After three years, 
staff turnover, changing circumstances, and new interests come to the fore. 

The culture of education in Alberta is changing. AISI has provided the means, the 
impetus, and opportunities for educators to become partners in their work. Funding 
provides the resources to make change possible. Annual workshops and conferences 
create opportunities for cross-district collaboration among coordinators, lead teachers, 
administrators, and others involved in changing education to improve teaching and 
learning. AISI reports document the knowledge gained through trying new approaches 
to teaching and learning, measuring the intended outcomes, reflecting on affective and 
behavioral dimensions of change, and adopting what works. 

AISI, currently in its third cycle, has demonstrated that it is sustainable. In the spirit of 
inquiry, collaboration, and continuous improvement, the initiative strives to continue to 
learn from its results and practices and to incorporate these lessons into subsequent years 
and cycles. Recommendations for subsequent cycles include continued focus on learning, 
ongoing support, knowledge mobilization, evidence, engagement, capacity building, and 
innovation and research (Alberta Education AISI, 2008). 



Policy 

What is the policy framework for AISI? The AISI goal, principles, and criteria have 
remained in effect since the inception of the initiative. This endurance over three cycles 
attests to the merit and acceptance of its policies and procedures. 

AISI policy is articulated in the AISI Handbook for Cycle 3 (AISI Education Partners 
Working Group, 2006). Initially there were two documents, the AISI Framework (AISI 
Education Partners Steering Committee, 1999) and the AISI Administrative Handbook 
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(AISI Education Partners Working Group, 1999). The Cycle 3 handbook incorporated 
both former documents and organized them so that the planning and reporting 
requirements were more clearly articulated and aligned. 

An initiative that has endured for nine years is being sustained. Elements that contribute to 
sustainability include the planning and reporting requirements that keep the focus on 
student learning; a changing culture in which inquiry is valued for the information it 
provides; collaboration among staff is becoming the modus operandi in place of teacher 
isolation; and teachers and administrators are working together to benefit both the major 
purpose of schooling - student learning - and their own professional needs and actions. 

The AISI approach to large scale improvement has also been adopted by other branches 
and divisions within Alberta Education. The Ministry now routinely involves education 
stakeholders in planning initiatives and deliberations. AISI has informed the strategic 
direction and supported the work of the literacy framework, the differentiated instruction 
manual, the high school completion initiative, the First Nations, Metis, and Inuit policy, 
the English as a Second Language initiative, and primary program planning. It has also 
served as the model for the Accountability Pillar Clearinghouse and promising practices, 
with hyperlinks to the AISI Clearinghouse. The Ministry is currently integrating lessons 
from AISI into policy and program considerations, communicating research findings 
widely, and promoting and celebrating classroom, school, and jurisdiction excellence 12 . 



Complexity 

The education system is complex. It involves multiple purposes, partners, processes, and 
outcomes. It adapts slowly to changes in society at large for a number of reasons including 
inertia, resistance, constraints, and standardization. Changing the culture is important, 
albeit insufficient. Change needs a purpose. Focusing on a goal - to improve student 
learning - and providing opportunities for teachers and administrators to enhance their skill 
sets - in projects addressing specific types of learning so they can integrate and sustain 
strategies that work - are ways to cultivate improvement in the education system. Collecting 
and using evidence are essential for supporting policies, practices, and decisions. 

AISI contributes to our understanding of innovation, renewal, sustainability, and 
resistance to change 13 . Resistance comes from all who are party to educational change: 
students, parents, teachers, and administrators. Change is the new constant. Keeping up- 
to-date becomes increasingly more challenging as advancing knowledge and emerging 



12 Strategy 2.4 in the Alberta Education 2008-11 Business Plan (Alberta Education, 2008, p. 9). 

13 See McEwen (2008) for a discussion of AISI’s innovation, renewal, and sustainability (including 
resistance to change). 



AISI Colloquium 



11 



Large Scale Improvement 



technologies continue to impact every area of life. Most of what baby boomers learned in 
schools is now outdated. The challenge to educators is one of embracing change - socially, 
culturally, technologically, economically, and politically. 

Three promising areas of research and development that can contribute to enhancing AISI 
are neuroscience, complexity thinking, and technology. 



Brain Research 

The human brain 14 is incredibly adaptive and estimated to contain about 100 billion 
neurons. The brain changes with learning, memory, and experience. Its plasticity is the 
result of many different, complex processes that occur in our brains. The brain constantly 
lays down new pathways for neural communication and rearranges existing ones 
throughout life. In response to a new experience or novel information, neuroplasticity 
allows either an alteration to the structure of already-existing connections between 
neurons, or forms brand-new connections; the latter leads to an increase in overall 
synaptic density, while the former merely makes existing pathways more efficient or 
suitable. The brain has a remarkable lifelong power to change. Positive or negative 
environments, exercise, nurturance, learning, and other experiences continue to change 
the brain throughout life. 

Neuroscience lets us explore the functioning of the brain. Neuro-imaging techniques 
such as Magnetic Resonance Imaging and Positron Emission Tomography scans can 
pinpoint which parts of the brain are involved in particular cognitive, affective, and 
behavioral functions. This type of information has enormous potential for improving 
teaching and learning. Neuroplastic research has shown us that every sustained activity 
ever mapped - including physical activities, sensory activities, learning, thinking, and 
imagining - changes the brain as well as the mind. Cultural ideas and activities are no 
exception (Doidge, 2007, p. 288). 



Complexity Thinking 

Complexity thinking refers to a philosophy for understanding complex phenomena and 
processes 15 . It differs from systems thinking in that it looks at wholes, rather than parts. 
Whereas systems thinking focuses on goals and objectives, plans, blueprints, and 
scenarios, complexity thinking views existence as a holistic manifestation of inseparably 
interwoven dynamics. It deals with concepts like strange attractors, fractals, self- 
organizing criticality, edge of chaos, etc., concepts foreign to systems thinking. The two 



14 The September 2008 issue of the School Improvement Scoop featured a number of online resources on 
brain research. 

15 This discussion derives from Dimitrov’s 2005 online article Complexity, chaos, and creativity, at 
http://www.zulenet.com/vladimirdimitrov/pages/complexthink.html . 
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differ in time frame as well: systems thinking is goal oriented, so it deals with the future: 
targets and outcomes. Complexity thinking focuses on the present. The challenge for 
complexity thinking is to understand the process of self-organization in society and to 
assist in eliminating the obstacles preventing its unfolding. 

Complexity and chaos explore the world as it appears to us: complex and chaotic - a 
wondrous manifestation of self-propelling creativity inherent in the wholeness of 
existence. The way creativity ‘speaks’ is through spontaneity of the emergence. And it 
is the emergence that is at the focus of complexity thinking, together with what makes 
this emergence happen: the intricate interplay of dynamics - forces and energies, which 
constantly stretch or fold, evolve or involve, self-organize into dynamic structures or 
dissolve into chaos impregnated with creativity. Complexity thinking nourishes and 
masters creativity, never trying to lock it into systems, subsystems and parts. And this is 
the greatest advance of complexity thinking. (Dimitrov, 2005, p. 7) 

Brent Davis and Dennis Sumara expand on innovation and complexity and some 
implications for AISI in Chapter 17. 



Technology 

Technology is another area with tremendous application to teaching and learning. Available 
24/7 (anytime, anywhere) people use a variety of technologies at home, work, school, and for 
commerce. Personal uses include networking, entertainment, learning, purchasing materials, 
and so forth. Most of us would be unable to work effectively without a computer, a number 
of software packages, and access to the internet. The web has made knowledge from around 
the world available at our fingertips. Technology has revolutionized how we communicate 
with one another, how we conduct research, and how we collect and analyze data. 

Today students differ from older generations in their attributes, likes, and dislikes. While 
older educators came to computers later in life, students have grown up with them as just one 
more technology at their disposal. Students are digitally literate, connected, experiential, 
social, visual, and kinesthetic. This new generation grew up with technology from infancy, 
which raises a number of implications for providing the net generation with an appropriate 
educational experience (Oblinger & Oblinger, 2005). 

Digital natives accustomed to the twitch-speed, multitasking, random-access, graphics- 
first, active, connected, fun, fantasy, quick-payoff world of their videogames, MTV, and 
internet are bored by most of today’s education, well-meaning as it may be. But worse, 
the many skills that new technology [has] actually enhanced (for example, parallel 
processing, graphics awareness, and random access) - which have profound implications 
for their learning - are almost totally ignored by educators (Prensky (2001) cited in 
Oblinger & Oblinger (2005, p. 2.14). 



Twenty-first century skills include core subjects, 21 st century content (global awareness, 
financial, economic, business, and entrepreneurial literacy, civic literacy, and health and 
wellness awareness), learning and thinking skills, ICT literacy, life skills, and assessment. 
Optimal benefits require a range of skills to harness the power of emerging technologies. 
The New Media Consortium (2005) defined 21 st century literacy as follows: 
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21 st century literacy is the set of abilities and skills where aural, visual and digital 
literacy overlap. These include the ability to understand the power of images and 
sounds, to recognize and use that power, to manipulate and transform digital media, 
to distribute them pervasively, and to easily adapt them to new forms, (p. 2) 

The characteristics of such literacy include multimodality, creative fluency and 
interpretive facility, learning a new grammar with its own rules of construction, 
interactive communication, ability to use media to evoke emotional responses, and 
potential to transform the way we learn (The New Media Consortium, 2005, p. 3). 

These characteristics apply equally to large scale improvement initiatives. For practice to 
be transformed it must operate in multiple modalities, involve fluency and interpretive 
facility, and be communicated in multiple ways. 

AISI can benefit from these three areas. Insights from neuroscience about how the brain 
functions offer many ways to improve learning. The power of complexity thinking may 
be in its use of questioning. Answers may have a short shelf life given the constant 
change surrounding us, but formulating cogent questions continues as a powerful way to 
focus our attention on key areas. And technology empowers us to work more effectively 
and efficiently. 



Transformation 

AISI is about more than three-year cycles and numerous projects. It is about changing 
education in Alberta. The initiative serves as a catalyst for improving teaching and 
learning based on evidence of what works well. Its focus on improving student learning 
is reinforced through the partnership and a culture of continuous improvement. Annual 
cycles of planning, implementing intervention strategies, and collecting and analyzing 
data to establish evidence of promising practices feed into the iterative processes of 
inquiry and reflection. Embedded in AISI is a strong component of building capacity in 
order to promote the sustainability of effective practices. 

As AISI matures and school authorities embrace inquiry, collaboration, and continuous 
improvement as the modus operandi of education in the province, it behooves Alberta 
Education to take the findings from the projects and meta-analyses of results (followed 
by in-depth research reviews of selected areas), and incorporate them into provincial 
policies and processes. The AISI research agenda 16 can provide impetus for such action. 

Four research priorities arise out of AISI’s strengths and recommended enhancements. 

The first three focus on student learning, instructional strategies, and building capacity. 
The fourth, infrastructure, identifies the foundation and support necessary to make the first 



16 School Improvement Branch. (2007, May). Mapping the future to improve student learning: A research 
agenda for AISI. Edmonton, AB: Alberta Education. The agenda elaborates on all of the priorities and 
provides an evidence base for them. 
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three happen; it is essentially how to optimize the benefits of AISI and consists of six 
elements: partnership, innovation, research, evidence, technology, and support. Figure 
1.3 presents the relationships. 




17 

Figure 1.3. Priorities for AISI’ s Research Agenda 



A transformed Cycle 4 incorporates what has been learned over the past decade of AISI, 
builds on its strengths, addresses its challenges, and aspires to continuous improvement. 
Such a transformation has five pillars: innovation, learning, instruction, teaching, and 
evidence. 



Innovation 

AISI systematically introduced innovation into education in Alberta. Innovation must 
continue as a foundation for future projects. At its best, innovation involves creativity 
and risk taking, novelty, and originality. Modestly interpreted, innovation means 
adopting approaches not currently used for the first time. 

There is no silver bullet or ‘one size fits all’ approach to school improvement. Innovation 
and choice differentiate AISI from other large scale improvement programs that aim to 
improve teaching and learning because it encourages people to try new things, to take 
risks. As the half life of information continues to decline, we must stay abreast of what is 
new and effective. Indeed the literature in areas such as neuroscience, technology, and 
complexity has much to contribute. 



17 

Source: School Improvement Branch (2007, p. 6). 
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Learning 



The goal of AISI is to improve student learning. Conceptions of learning continue to 
evolve including its multidimensional nature (cognitive, affective, and behavioral), social 
and emotional learning (CASEL) 18 , the theory of multiple intelligences 19 (Gardner, 

1999) , dimensions 20 (Covey, 2004), and pillars (Delors et al. 21 , 1996). 

We know that learning must be fun for children (and adults) to continue. If they are 
active and engaged, and able through experience to discover what interests them, they 
will continue to put in countless hours to master knowledge and skills. Brain research 
tells us that emotion is the gatekeeper to learning, intelligence is a function of experience, 
and personal meaning is the key to memory. See for example McGeehan (2001), 

Johnson (2006), Doidge (2007), and Aamodt and Wang (2008). 

What should students learn? Alberta’s Programs of Study outline curriculum content 22 . 
Lifelong skills are articulated in the Conference Board of Canada’s (CBC) employability 
skills 2000+, which include fundamental, personal management, and teamwork skills (CBC, 

2000) and innovation skills which include creativity and continuous improvement, risk 
taking, relationship building, and implementation skills (CBC, 2003). And 21 st century 
skills include core subjects, 21 st century content, learning and thinking skills, ICT literacy, 
life skills, and assessment. Optimal benefits require a range of skills to harness the power of 
emerging technologies. 



18 CASEL is Collaborative for Academic, Social, and Emotional Learning (Durlak & Weissberg, 2005). 

19 Linguistic, logical-mathematical, musical, bodily-kinesthetic, spatial, interpersonal, and intrapersonal. 

In 1999, Gardner added an eighth - naturalist intelligence which enables human beings to recognize, 
categorize and draw upon certain features of the environment. A multiple intelligences perspective can 
enhance understanding in at least three ways: by providing powerful points of entry, by offering apt 
analogies, and by providing multiple representations of the central or core ideas of a topic (Gardner, 1999, 
pp. 186-187). 

20 Mind (mental), body (physical), heart (emotional), and spirit (spiritual). 

21 Delors et al. 21 (1996) envisaged education throughout life to be based on four pillars: learning to know, 
learning to do, learning to live together, and learning to be. Jacques Delors was the chair of the International 
Commission on Education for the Twenty-first Century, whose final report was entitled, Learning: The 
treasure within. 

22 Alberta’s Early Childhood Services (ECS) to Grade 12 curriculum is outlined in provincial programs of 
study, which identify what students are expected to learn and do in all subjects and grades. The curriculum 
is designed to help students achieve their individual potential and create a positive future for themselves, 
their families, and their communities, http://education.alberta.ca/teachers/program.aspx . 
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Instruction 



The section on school improvement (pp. 3-4) identifies a number of research studies on 
instructional strategies that work. Instruction can be differentiated by content, process, 
product, environment, learning styles and modalities, learning strategies (e.g., inquiry, 
constructivism, collaborative), instructional design (e.g., problem based, experiential, 
independent study), and technology (e.g., flexibility, personalized learning, 24/7). 
Teachers must embrace the literature and research on instruction to increase their 
repertoire of skills. 

There are many sources of information about what works. These include the What 
Works Clearinghouse , the Campbell Collaboration , the Organization for Economic 

9 S 9 

Cooperation and Development , the Council of Ministers of Education, Canada , and 
the Rand Corporation 27 . 



Teaching 

In schools, teaching will remain the predominant way to help students learn. Teachers 
who inspire students to be the best they can be are the ones we remember long after 
formal schooling has ended. Teachers need to be flexible and adaptable to accommodate 
learning experiences to suit the students in their classes. By collaborating with their 
colleagues they expand their knowledge and skills and can serve as a critical friend or 
mentor. If relationships truly matter in helping students learn and teachers become better 
practitioners, then building relationships with students, staff, and the community are 
important skills. 

Embedded in teaching is the ongoing commitment to building capacity. In today’s 
knowledge economy, where information is estimated to double every 1 8 months, there is 
a need for people to continue their professional development throughout their careers. 

For this reason, it is imperative that capacity building be intentional, ongoing, and 
systemic. Alberta’s Teaching Quality Standard (Alberta Education, 1997) applies to 
teacher certification, professional development, supervision and evaluation, and is 
supported by descriptors of selected knowledge, skills and attributes (KSAs) appropriate 
to teachers at different stages of their careers. 



23 US Department of Education Institute of Education Sciences at http://ies.ed.gov/ncee/wwc/ . 

24 http://www.campbellcollaboration.org/ . 

25 http://www.oecd.Org/topic/0.3373.en 2649 37455 1 1 1 1 37455.00.html . 

26 http://www.cmec.ca/index.en.html . 

27 http://www.rand.org/research areas/education/ . 
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People learn by doing and become more skilled as they gain experience. Expert knowledge 
is more sophisticated than that of novices. Experts’ knowledge is characterized by 
meaningful patterns of information, organization of knowledge, context and access to 
knowledge, fluent retrieval, and adaptive expertise (Bransford et al., 2000). 

Teachers are also learners. In the words of a seasoned AISI coordinator: 

The best teachers are those who are supported as learners themselves. The encouragement 
and affirmation of teachers as learners of best practices that has been a key part of AISI is 
creating an environment where teachers can thrive. Our students deserve the best teachers 
and AISI is helping them get what they deserve. (Gertz, 2007, p. 3) 



Evidence 

If innovation, learning, instruction, and teaching are the cornerstones of school improvement, 
then evidence is its keystone, or lifeblood. Without evidence, in its many forms, we do not 
know if we are making a difference. 

• Has the innovation been successful? How do we know? 

• How do we know that students learned? What impact did an intervention have on 
different types of students? 

• What evidence do we have that intervention X had an impact? Was its impact better 
than intervention Y’s? 

• Which aspects of teaching were most effective? Why? 

These are some of the questions that evidence can answer. Education shares with medicine 
and justice a need to use evidence in making decisions. Medical doctors use evidence to 
determine whether a particular approach is more appropriate, under what conditions, and 
for which types of patients. The justice system uses testimony, exhibits, and other 
documentary material to establish guilt or innocence. So too should education use the 
evidentiary tools at its disposal. 

Multiplicity is a hallmark of evidence. Multiple perspectives (e.g., students, parents, 
teachers, public), multiple methods and multiple data sources (e.g., assessments, surveys, 
observations, administrative data) and multiple levels (e.g., elementary, secondary) assist in 
triangulation. Longitudinal study and the use of effect sizes help to judge the value of an 
intervention. Robert Crocker discusses research approaches in depth in Chapter 14. 

We live in an information age in which language helps us to express, clarify, and organize 
our thoughts. Signature activities include reading, writing, computer literacy, and using 
electronic media. Signature activities require training and cultural experience and lead to 
the development of a new, specially wired brain. Brain plasticity allows us to adapt to a 
vast range of environments (Doidge, 2007, pp. 290-291). 
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An important aspect of any type of inquiry is reflecting on what is learned and documenting 
processes and findings. Writing is an effective way to make sense of what is happening. 
Ideas can be ephemeral; if unrecorded, they are often lost. To paraphrase Plato’s “the life 
which is unexamined is not worth living” and Santayana’s “those who cannot remember the 
past are condemned to repeat it”, unexamined (and undocumented) ideas and actions are 
doomed to be forgotten. Organizing and interpreting data, ideas, and insights into a variety 
of meaningful documents not only preserves them for posterity, but uses them as a 
springboard for continued action and improvement (McEwen, 2007, p. 16). 

Documentation can take many forms. The most common is a written record (through 
reports, articles, and books), but other ways include videos and stories. 



Conclusion 

The three cycles of AISI have been remarkably successful in bringing about change in 
education in Alberta. AISI demonstrates that large-scale innovation in a province’s 
education system is possible. Change is taking place at all three levels of the system: 
school, district, and province. At the provincial level, the partnership, online database 
system, financial and other supports continue to sustain the initiative. AISI provides a 
clear opportunity to try new things, to take risks. At the district level, school authorities 
can be as creative and innovative as they wish. They have choice and flexibility in 
designing projects that meet local needs. School staffs have become much more 
collaborative in finding effective ways to help diverse students meet their learning needs. 
And participants in AISI - partners, teachers, administrators, and others - are actively 
promoting the knowledge generated by AISI in the service of improving teaching and 
learning. 

AISI models how a large-scale reform initiative can be sustained by providing leadership, 
continued funding, and ongoing support. The initiative also demonstrates how renewal is 
built into changing the culture of schools to achieve the goal of improving student learning. 
Targeted funding provides the means through which staff can engage in capacity building 
activities, both in the school and at external events such as workshops and conferences. 
People learn by doing. Teachers are learning to expand their instructional repertoires to 
engage students and meet the increasing diversity of student needs. Students who are 
actively engaged learn more effectively than those who are not. Renewal is also important 
in accommodating staff changes and refreshing the initiative itself. 

But AISI must not become complacent. Cycle 4 cannot be just more of the same. If AISI 
is to remain on the leading edge, it must continue to innovate, to focus on student learning, 
and to document how and why instructional strategies and teaching approaches positively 
impact student learning. An AISI transformed must aspire to higher levels of innovation, 
student learning, effective instruction, and teaching practices. It must incorporate emerging 
knowledge and technologies and document what works. 
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The AISI Colloquium, a historic meeting of three groups - partners, school authorities, 
and external experts - provides the impetus to discuss AISI’s strengths, ways it can 
improve, and to imagine how it can be transformed. The collective wisdom of the group 
will go a long way to imaging a preferred future for AISI. 
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Chapter 2 



Has AISI Run Out of Gas? 

Heather Welwood 

Alberta School Boards Association 



I believe we are here to be thought provoking so today I am going to ask you to consider 
the following question: Has AISI run out of gas? AISI - the Alberta Initiative for School 
Improvement - was born in December 1999. It was the offspring of a frantic - yet much 
enjoyed - collaboration of six 28 enthusiastic partners: 

• Alberta Learning 

• Alberta School Councils’ Association 

• Association of School Business Officials of Alberta 

• College of Alberta School Superintendents 

• Alberta Teachers’ Association 

• Alberta School Boards Association 

This year marks the end of Cycle 3 of AISI. Just think of it, three consecutive three-year 
cycles of AISI initiatives. Our pride and joy is nine years old! And hasn’t she grown up 
to be a real beauty! Of course she has. After all AISI represents about half a billion 
dollars of taxpayer dollars over those nine years - an investment that has catapulted 
student achievement through the roof! ! Well maybe not quite. In fact during AISI’s 
entire life as the “showpiece” of Alberta’s education system, when we just look at our 
student achievement results - as measured by our Provincial Achievement Tests (PAT) 
and Diploma Exam results - they have remained flat or in some measures, modestly 
declined. Our high school completion rates are also fairly flat. What would an auditor 
looking at value for money tell us? 

PATs and diploma exams don’t tell us everything about student achievement in our 
schools, but I’m sure that everyone in this room will agree, they do tell us something 
about student achievement at the jurisdiction and provincial levels. Staff members are 
changing how they learn, assess, and teach, but student learning is at the top of the 
pyramid and our ultimate goal. 



28 

Editor’s Note. University Faculties of Education (Alberta, Calgary, Lethbridge) were invited to join the 
partnership in spring 2000. 
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What is AISI? 



So what is it about AISI, this peripatetic paragon of provincial pedagogical programming? 
Have we arrived at the point where AISI has just become a part of the family and the plans 
and excitement we had for our newborn have been lost in everyday living? We pampered 
and planned for our first born, talked, took pictures, and bragged. We welcomed the second 
child too, but with a little less ceremony and attention. By the time the third child came 
along, we just weren’t as enthusiastic. New excitement, recordings of life, and enthusiasm 
were lost but no one dares to admit that the third child, who is cared for and loved as much 
as the first, didn’t get the same treatment! Every organization involved in AISI supports the 
initiative, and the organization that I serve - the Alberta School Boards Association - is 
certainly no exception. 

And while we can all be deluded into group think from time to time, I’m convinced that 
the core of AISI success, and its reputation, rests on the circumstances surrounding its 
birth some nine years ago. 

After all else is cast aside, the AISI secret is collaboration, spelled in great big 30 point 
font characters. Collaboration among teachers, school boards, parents, central office 
staff, and Alberta Education. In a world that has become characterized by top-down 
directives, AISI stands out as “bottom-up” governance. 

AISI works because those in the field all want it to work. We want to believe that we can 
make a difference and we do because everyday wonderful things are happening in Alberta’s 
classrooms. Many of these wonderful things are as a result of AISI. 



Making AISI Better 

But what I would like to say to you forthrightly this morning is, “I believe we can, and 
must, do better with AISI.” AISI is just too good a program to let drift into yet another 
bureaucratic exercise in school improvement. And the challenge I place before you is: 
How do we best do this? How do we rekindle the promise of AISI? How do we breathe 
new life into this precious initiative? 

Now that surely is the central question at this colloquium. I hope you’ll allow me a few 
more minutes to muse about some of my own thoughts on this question. I have three to 
share with you. 

1. Risk. AISI has become “risk adverse.” We need to find a way to encourage school 
jurisdictions to stretch the envelope again. You know the saying “no pain - no gain”. 
I believe this is true if we are to springboard Alberta’s education system - already one 
of the best in the world - to the next level. We need to be bold and innovative. We 
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need to recognize that with good research and change, there may be failures along the 
way. We must balance the need for innovation with the reality that with innovation 
there sometimes comes the risk of failure. But I believe that’s what AISI is all about, 
finding ways to move our student achievement forward. 

2. Focus. I believe that all school jurisdictions should target their AISI projects in areas 
where they are struggling to find ways to improve. Our province is facing two 
educational issues: improving high school completion and/or the achievement gap 
facing our First Nations, Metis and Inuit students. All jurisdictions can and must do 
better in these areas. Our current results are just not acceptable. Just think of the 
potential gains if all jurisdictions targeted their AISI projects towards these challenges 
according to their own unique local needs. 

Imagine some bold and innovative projects for school boards. It may come as a 
shock to some people here that many of our teenagers have trouble getting up in the 
morning (actually some of our trustees and staff do too). I am told brain research 
confirms this is because teens’ brains work differently. Could an AISI project which 
recognizes this reality and designed the school day around the biological needs of 
students with programming at different hours, six or seven days per week make a 
difference in our high school completion rates? In my books a project like this is 
definitely worth a try. 

3. Knowledge Transfer. Finally, I want to speak about the need to improve how we 
share and learn from one another. AISI makes a number of efforts to transfer 
information gleaned from one school jurisdiction to another. We need to get much 
better at the science of “knowledge transfer”. This is not a problem confined to AISI; 
it is endemic to our entire education system. It takes time to share with everyone and 
we have turnover within our systems. We need to find a way to make sure that AISI 
initiatives that are successful in one jurisdiction - those gems of brilliance that are 
shimmering out there - are shared and implemented in others, with the necessary 
changes to adapt to local circumstances. After all, if our teachers, administrators, and 
school trustees don’t have the opportunity to learn from what is successful in other 
parts of the province, how can we expect the public to know their system is being 
managed well and we are continually striving for improvement? 

Now those are just my thoughts - not those of every school district in the province. I’d be 
remiss in my duty if I didn’t tell you that many school boards tell me there is too much 
paper work, too much bureaucracy, and the project cycle needs to be longer. I don’t believe 
that is why we are here for the next three days. We need to focus on the big picture and that 
is why I have tried to be thought provoking. I hope this colloquium takes up the challenge 
of not just praising AISI, but of critiquing it, of showing us how we can get better bang for 
our buck, how we can do it better, and do it more efficiently. 
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Let this AISI Colloquium and a subsequent meeting of the original partners present at the 
birth, review our discussions and set the scene for a revitalized and reborn AISI. An AISI 
that is innovative and takes risks, focuses our efforts on our key problems, and one which 
fosters sharing and learning transfer among all partners. Best wishes for a successful 
colloquium and thank you to all the people who have made it possible. 



Heather Welwood is President of the Alberta School Boards Association. She can be reached at 
heather. welwood@nlsd.ab.ca. 
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Chapter 3 

ASCA Thoughts on AISI - Past, Present, and Future 



Tina Boymook 

Alberta School Councils’ Association 



I’d like to thank the organizers of this event for the opportunity to share the perspective 
of the Alberta School Councils’ Association. For those of you who aren’t familiar with 
ASCA, we are the provincial association whose mandate is twofold - we bring the 
parents’ perspective on public education matters to Alberta Education and others, and we 
provide service and support to school councils to enhance their effectiveness. 

As one of the original education partners involved in its creation, we have watched the 
Alberta Initiative for School Improvement (AISI) grow and evolve over the past nine 
years. As we have been involved on the provincial level on the AISI Education Partners 
Steering Committee, we’ve noted some things along the way. We’d like to share those 
observations and propose ideas that may help inform AISI of the future. 

In the beginning AISI was considered innovative - innovative because it was developed 
collaboratively by education partner organizations (unheard of!); innovative in concept 
(nothing like this had been tried anywhere else); innovative in design (project based with 
no loss of funding if the project wasn’t as successful as one hoped). Innovation was built 
into AISI at every level - projects were to contain an element of risk and were encouraged 
to be different and original. As a result, projects varied from school to school, reflecting 
the uniqueness of school communities, addressing student or classroom challenges in 
original ways. 

In the beginning those of us who were involved said we must guard against AISI 
becoming taken for granted or becoming stale over time. We said we must guard against 
AISI becoming all about funding - or becoming just another initiative to free up budgets 
for other things. Most importantly we felt we must guard against AISI losing its 
innovative and risk-taking nature. 

We believe AISI has drifted from its original concept - innovation, risk taking, projects 
that have the potential to engage all members of the school community in school 
improvement strategies. We sense its drift away from its original concepts and its drift 
toward being seen as another funding source. Nine years out, is AISI still achieving what 
was intended, what was envisioned? 
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We know AISI has made a difference. We know we don’t want to lose an important 
initiative that has opened many doors of change and opportunity for students and their 
learning. 

And we also feel after nine years, it is time to critically examine AISI - discuss amongst 
ourselves, the original partner organizations - whether AISI needs a tweak here and 
there, or a complete overhaul. Can there be a new and improved AISI? We think so. 
Cycle 4 is almost here, and this colloquium provides a great opportunity to look ahead. 

Let’s look ahead, not only to Cycle 4, but to a potentially new iteration of AISI. Together, 
let’s ensure AISI is doing what was originally intended - fostering innovation in Alberta’s 
classrooms, nurturing a culture of change, opening doors for collaborative efforts for all 
education stakeholders in a school community, and most importantly, making a positive 
difference for Alberta’s students. 



Tina Boymook is Past President of the Alberta School Councils’ Association. She can be reached at 
parents @ albertaschoolcouncils.ca . 
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Chapter 4 

Getting to the Heart(h) of AISI 



J-C Couture 

Alberta Teachers’ Association 



Without question, the Alberta Initiative for School Improvement (AISI) has been a vital 
part of the education landscape in Alberta for the past nine years. An important clue to 
explaining its success is contained in the work of the social philosopher Albert Borgmann 
(1992), who coined the term “focal practice.” For Borgmann, undertaking professional 
work in an increasingly complex world involves not only engaging emotionally in the 
many invisible tasks that a professional carries out with skill but also responding to the 
moral imperative to make a positive difference in the lives of other people. 

Borgmann reminds us that the word “focus” derives from the Latin word hearth - a 
gathering place that brings people together in a convivial way for a common cause. 

These places may be as mundane as the local hockey rink in the midst of a cold Alberta 
winter. In cases of focal realism, the “rich reality . . . sponsors a sense of community” and 
becomes a place where “reality and community conspire .. . .” (p. 135). 



Professional Learning Communities 

Viewing the professional learning communities that exist in our schools as “conspiring 
communities” rather than as arenas of accountability and continuous improvement (to use 
some of the latest buzzwords) helps us to appreciate their power and warmth as points of 
“focal practice”. 

Borgmann points out the enormous difference between the central heating plant that 
mechanically warms most modern buildings and the fireplace or hearth (p. 42) that served 
in earlier times as a meeting place. The central heating plant is a machine that removes 
the need for people to come together to keep warm and, in so doing, weakens the sense of 
community. The hearth or fireplace, by contrast, becomes a natural place for people to 
socialize. By eliminating the need for people to come together to build and tend the fire, 
the central heating plant diminishes community and conviviality. Borgmann argues that 
it is not the fire itself but the conversations that take place around the fire that produce 
community and help meaning to emergence: “A [focal] practice keeps faith with focal 
things and saves them for an opening in our lives” (p. 209). It is by bringing teachers 
together to consider ways of improving student learning that AISI represents a focal 
practice. 
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As the literature demonstrates, school improvement can easily deteriorate into empty 
sloganeering, an approach in which catchphrases substitute for real meaning and 
significance in the lives of students and teachers (Hargreaves & Fink, 2006). For these 
writers, authentic professional inquiry thrives only if it is accompanied by a comprehensive 
set of school-level supports, including depth and conservation strategies. Depth strategies 
attempt to bring about real (or deep) learning by demonstrating that inventiveness and 
ingenuity are fostered by slow learning that focuses on locally determined priorities and 
needs. Real learning, in other words, is connected to the emotional engagement as well as 
the intellectual growth of teachers. 

The literature on organizational improvement underscores time and again that “it is about 
learning as a community” (Stoll, Fink, & Earl, 2003, p.132). Collaborative-practitioner 
research in a learning community is more than a clever turn of phrase; in order for schools 
to foster learning, community must come first. We must think of the many possibilities of 
learning and community if we are to succeed in our goal of “learning of community, 
learning from community, learning with community, learning for community, and learning 
as community” (p. 134). 

If learning of/from/with and for community is the fulcrum for school development, 
focally real teacher research is possible only by undertaking school improvement projects 
that respond to the immediate realities of students’ learning needs as determined by local 
communities of practice. Just as important, teacher research must include opportunities 
to influence and shape the contexts of teachers’ work. A surgeon friend of mine recently 
told me that she learned from her early practice “that we cannot skill our way out of 
limitation.” She became increasingly frustrated by her own early “ham-handed attempts” 
to tie off veins and arteries. 

AISI has made professional communities of practice possible, and it needs to continue to 
value and nurture them in Cycle 4. Borgmann’s characterization of focal things and 
practices reminds us of the power of sustained collaborative inquiry among groups of 
committed teachers focused on a common learning challenge. 

[We] might say this about focal things in general. They are concrete, tangible and 
deep, admitting of no functional equivalents; they have a tradition, structure, and 
rhythm of their own. They are unprocurable and finally beyond our control. They 
engage us in the fullness of our capacities .... A focal practice, generally, is the resolute 
and regular dedication to a focal thing. It sponsors discipline and skill which are 
exercised in a unity of achievement and enjoyment, of mind, body and the world, of 
myself and others, and in a social union. (Borgmann, 1992, p. 219) 

In focal things and the practices that guard them, reality is experienced as continually 
commanding the attention of teachers who collaborate to improve their practice. Teacher 
research provides an opportunity for teachers to become “focally real” in ways that will 
help them make a real difference in their students’ lives. 
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Thanks in large part to AISI, teaching as an intellectual and theoretical pursuit can no 
longer be discounted the way that it sometimes is in other jurisdictions (Darling- 
Hammond & Sykes, 1999). Cycle 4 of AISI must sustain the “patient vigor” of focal 
practice. Reading through the hundreds of AISI project reports and presentations by 
teacher researchers, one cannot help but be struck by a compelling image - that of 
teachers coming together to ponder practical ways of improving student learning. In a 
recent joint publication by the Alberta Teachers’ Association and the Alberta Assessment 
Consortium (2008), for example, teachers in four schools from across Alberta dedicated 
an entire school year to exploring ways of improving their approach to assessing students 
in the early years of learning. During their monthly meetings, these teachers focussed on 
the case study of one student for the entire year. The power of this single-minded focus 
brought a sense of both urgency and clarity to their work. Nettlesome questions such as - 
What, if anything, does a grade level mean in a course with 200 outcomes? and At what 
age is it appropriate to introduce students to rubrics? - began to surface. Yet, in large 
part because these teachers were dealing with a single student rather than an abstract 
concept, they refused to walk away from such difficult questions, persevering until they 
had found a solution. 

In that project, the focally real practice of concentrating on an individual student 
produced an approach to assessment that recognized the inherent difficulty of teaching - 
what Dunne (1993) refers to as taking teaching “back to the rough ground” where smooth 
trekking is not at all guaranteed. The project reminded teachers that the biggest threat to 
school improvement efforts is almost certainly the appropriation of techniques that focus 
on such simplistic goals as getting quick results on test scores. Coffield (2007) describes 
how these “insidious processes” took hold in England, which ran “ever faster down the 
wrong road” in its efforts at school reform. Smits (2003), by contrast, argues that we 
need to be continually “provoked” into returning our focus to the individual student. In 
his view, “to be provoked is, literally, to be called or summoned by something.” From 
this perspective, individual students act as buffers, preventing teachers from “passing 
judgment too quickly,” which is one definition that the Oxford English Dictionary 29 
offers for the word “provocation.” 



Teacher Professional Inquiry 

Over the past nine years, AISI has helped educators move past the age-old debate over 
whether teacher professional inquiry and action research constitute real research. Yet, 
for reasons that have to do with power and economic interest, this tiresome question 
refuses to go away (Lincoln & Cannella, 2004; Maxwell, 2004). Popkewitz (2004) 
argues that, against the growing neoliberal tide that focuses on “best practices” and 
narrowly defined limits of “what works,” teacher research is increasingly marginalized by 
so-called scientific critiques that have nothing to do with science (p, 63). The criticisms 



29 1984 edition, p. 2341. 
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typically levelled against teacher research focus on the lack of scalability (i.e., Can this be 
replicated?) and validity (i.e., Did you consider other factors that may have influenced the 
outcome?). Although these are worthwhile questions, the focally real practice of teacher 
collaborative inquiry sustained through AISI returns teachers to the “rough ground” where 
they are forced to grapple, afresh, with the original difficulties of teaching. 

Hargreaves (2003) writes that teachers exist in a paradoxical position between, on the one 
hand, the promise of living in the knowledge economy and, on the other, the stark realities 
of underfunded schools and the tendency of policy makers to default to “standardized 
solutions” to address complex educational issues (p. 10). Hargreaves argues that teacher 
inquiry and research play a crucial role in determining whether professional learning 
communities will become a reality or simply an empty slogan. 

Cycle 4 of AISI needs to continue to trust in the capacity of teachers to respond to students’ 
learning needs. The emergence of focally real communities of practice may enable us to 
move away from such dated approaches as standardized tests, managerial supervision, and 
government compliance requirements and legislation (Hargreaves, 2006, p. 50) toward a 
focus on school-based and community-level accountability. 

Student learning and success are at the heart of meaningful school improvement. As 
Reeves (2006) puts it, “If we are to learn from our mistakes, then we must begin our 
journey with an acknowledgement that we have an educational Enron waiting to happen 
if we fail to recognize that the demands of leadership are more complex than intimidating 
students and teachers into short-term test score gains” (p. 3). 

All research endeavors - and AISI is just one example - are made possible for political 
reasons (Herr & Anderson, 2005, p. 68). All education partners, including teachers, must 
acknowledge this reality and continue to advocate for the resources and the embedded 
professional development opportunities that will foster improved student learning. 

Viewed in this way, Cycle 4 of AISI promises to sustain teacher research as a focally real 
practice. 
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Chapter 5 

Experiencing AISI from a School Business Officials Perspective 

Christine L. Lee and Alberta Hutchings 
Association of School Business Officials of Alberta 



To understand a school business official’s perspective on AISI, we would like to provide 
some insight into school business officials. The Association of School Business Officials 
of Alberta (ASBOA) is a professional organization dedicated to supporting our members 
and providing leadership in school business management. Our members are involved in 
public policy, legal matters, risk management, human resources management, facility 
planning, construction and operations, purchasing, strategic planning, information 
management, communications, transportation, and fiscal management and accountability. 

Our association holds a number of beliefs, some of which specifically fit with the principles 
of AISI. 

• The integral participation in the provision of quality education services. 

• Establishing and maintaining high standards of efficiency and effectiveness in school 
business methods and practices. 

• Encouraging research for the improvement of school business management practices 
and sharing of best practices. 

• Providing professional development opportunities for our members. 

• Providing opportunities to all members to encourage the exchange of information and 
ideas. 



Observations and Experiences 

AISI has had a profound impact on improving teacher practice and student learning over 
the last three cycles. The success of AISI is attributed to a process of school improvement 
that has been shared by all stakeholders. 

It is our association’s observation and experience that there have been three areas that 
have been key to the success of AISI. 

Partnership. An African proverb states that it takes a village to raise a child. The AISI 
partnership brings education partners together on a collaborative level to work towards 
improving student learning in our province. Our association has been pleased to participate 
in the AISI Education Partners Steering Committee as one of our beliefs is to be an integral 
part of the quality provision of educational services. The success of AISI is attributable to 
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“the whole village” being involved in improving teaching and learning in Alberta. AISI is 
a fabulous partnership exemplifying shared vision, trust, and commitment. It has 
maintained these elements even with the changing of individual representatives. 

Flexibility. AISI has allowed jurisdictions to initiate projects based on local circumstances. 
Without tapping into “everyday” instructional funding, schools and jurisdictions have been 
able to become “risk takers” and “pioneers” to create unique projects that will improve 
student learning. AISI has provided the mechanism for local community engagement in 
developing learning opportunities for students. 

Best Practices. The sharing of information has facilitated a culture of collaboration and 
continuous improvement in and among school jurisdictions. This includes sharing ideas 
and best practices among school-based teachers, school-to-school sharing in a jurisdiction, 
and jurisdictional sharing through the AISI Clearinghouse, the AISI conference, and 
networking. ASBOA believes in encouraging research for the improvement of school 
business management practices and sharing best practices. AISI has provided the 
opportunity, through research and evidence-based data gathering, to share successful 
strategies that have led to improved student learning through Cycles 1 to 3, and is one of 
the cornerstones of the program that has received world-wide acclaim. 



Enhancements for Cycle 4 And Beyond 

AISI has had a powerful impact on school jurisdictions. Administrators, students, 
parents, and teachers are more engaged in the learning process and there is a common 
language and focus at all levels which has led to a professional dialogue. There is need 
for continuous performance improvement as we evaluate previous cycles and make 
modifications that will increase the efficiency and effectiveness to enhance student 
learning in Cycle 4 and beyond. ASBOA would like to recommend the following 
enhancements for AISI Cycle 4 and beyond. 



Funding 

Over the past three cycles, it has been observed that it may take until the middle of a new 
cycle to finalize projects and appropriately plan for a new cycle of AISI. If boards have 
the flexibility to complete Cycle 3 projects while planning new projects for Cycle 4, then 
there will be better transition between projects. Some boards find it difficult to complete 
projects by the end of a three-year cycle. 

Since school jurisdictions are provided total funding based on the September 30 th count 
of the prior year, it would be appropriate that funding be released as soon as a project is 
submitted. Currently funding is released based on approval, which can take several 
months after a project has begun. 
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Project Life Cycle 

In order to appropriately measure systematic change, more than three years may be 
necessary to gather appropriate research and to adequately report on the success or failure 
of a particular project. If school jurisdictions are able to provide appropriate evidence 
that a Cycle 3 project requires more time, then flexibility should be provided to continue 
that project and to provide appropriate evidence of its success or failure. 

A longer life cycle gives teachers the time to talk about their practices, trust the research, 
and build strategies into their practice. Change is required at other levels beyond the 
classroom. Some strategies require changes at the school level, others at the district 
level, and acceptance and understanding are required by parents and other members of 
the community. Because of the three-year cycles of AISI, once a project is over, it may 
be difficult for a school to maintain the professional development that is required to 
increase sustainability. 

Three years is a very short time to create the necessary desire for such changes to be 
implemented. AISI is a worthwhile feature of the education system in Alberta. Possibly 
five-year cycles would provide the required time to make real organizational changes. 



Reporting 

It is worthwhile to capture the essence of the research undertaken through AISI, but some 
of the reporting is seen as compliance reporting. It is difficult to see the purpose of some 
of this reporting from the school jurisdiction perspective. It may be more appropriate, 
from our standpoint, to report on outcomes rather than inputs 30 . Some jurisdictions might 
only pursue one AISI project per cycle simply because of the level of reporting required. 
The level of reporting may have prevented jurisdictions from considering small 
meaningful projects. 



Beyond Cycle 4 

Assurance that a new cycle will begin is required at a minimum of 12 months in advance in 
order for school jurisdictions to adequately gather information and research on what 
strategies should be employed that may have the greatest impact on improvement in the 
jurisdiction. School authorities require time to employ appropriate research methodologies, 
collaborate with all stakeholders, and have the opportunity for inquiry and reflection on 
past projects in order to develop effective projects for a new cycle. 

It is important to continue to encourage jurisdictions to try innovative projects that may 
not result in guaranteed success, but result in valuable information for future strategies. 



30 Editor’s note. School authorities are required to submit proposals for AISI projects that meet specific 
criteria. Once a project is approved, school authorities must report annually on how well the project met 
these criteria. 
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Conclusion 



Covey (1991) stated that an empowered organization is one in which individuals have the 
knowledge, skill, desire, and opportunity to personally succeed in a way that leads to 
collective organizational success 31 . AISI has certainly been an engine of empowerment 
to school jurisdictions to provide successful learning opportunities for students in 
Alberta. 

ASBOA enjoys being an integral part of the AISI experience through our involvement in 
the AISI Education Partners Steering Committee. We appreciate the opportunity to 
comment on this worthwhile initiative that has enhanced teaching and learning in our 
province. 



Christine L. Lee is Past President of ASBOA. Alberta Hutchings represents ASBOA on the AISI Education 
Partners Steering Committee. They can be reached at christine.lee@lethsd.ab.ca and albertah@eics.ab.ca . 



31 



Covey, S.R. (1991). Principle-centered leadership. New York, NY: Fireside Press. 
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Chapter 6 

AISI: The College of Alberta School Superintendents Perspective 

Paulette Hanna 

College of Alberta School Superintendents 



Observations 

We are pleased to capture a broad perspective on the Alberta School Improvement 
Initiative (AISI) from feedback we have received from system education leaders. 

Investment. AISI is a tremendous investment in the education of our students. Our 
appreciation goes out to Alberta Education and our Minister for supporting this initiative. 

Teacher Opportunities. AISI has had a positive impact on developing professional 
learning communities. It has opened the doors of classrooms, encouraged shared 
practice, supported school-based educational leadership, and provided strategic, planned, 
and focused professional development. 

The greatest impact of AISI on schools has been on the growth of teacher knowledge, 
practice, and collaboration. The “olden days” where teachers closed the doors of their 
classrooms and forgot about the rest of the school are long gone. The synergies among 
teachers across grades, divisions, schools, and districts have increased exponentially. A 
supported, resourced, respected teacher, who is given the opportunity to grow 
professionally, is the key for making a difference for students. 

Improved Student Learning Opportunities. While research should be supportive of 
initiatives, what we do should not solely be based on quantitative results or we have lost 
the spark of innovation in our projects. The opportunities of the many varied projects has 
enhanced student learning, often taking students beyond academics. Accountability and 
the decision making that are inherent to AISI projects power data-driven decision 
making. 

Strategic Planning. AISI projects are becoming more and more integrated into school 
system strategic planning to the point that some of our leaders feel it shouldn’t be an 
attachment to the basic plan. On the other hand, the extensive application and reporting 
requirements (targeted funding) would be difficult to sustain if AISI was fully integrated. 
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Change and Improvement. AISI was initiated to spark creativity and innovation to 
further school improvement. Over the past nine years AISI funding has supported change 
for improvement in our schools. The AISI Knowledge Dissemination and Integration 
Plan summarizes the impacts throughout the province. 



Concerns 

Our concerns relate to innovation and risk taking, administration and reporting, sustainability, 
and impact. 

Innovation and Risk Taking. Is AISI as innovative as it was in the beginning? The 
movement from school-based to system-based projects is logistically more manageable 
and measurable but does eliminate smaller projects. CASS would hesitate to agree to 
further broadening of the projects. Research should support but not be the whole 
foundation of AISI projects or the attempts to measure projects will eliminate risk taking. 

Administration and Reporting. AISI seems to be “institutionalized” into bureaucracy 
with onerous paperwork. CASS members feel that the paperwork for AISI must be 
looked at from the perspective of expectations on systems. Administration costs should 
be allowable in AISI budgets. AISI is highly appreciated but does put pressure on school 
system resources. We should review the approval process and reporting requirements. 

Sustainability. The impact on school systems would be huge if funding was withdrawn. 
Sustainability of projects would be very difficult without significant changes to allocations 
to schools. CASS leaders would appreciate assurances of ongoing AISI funding. 

Impact. Student achievement on provincial tests and exams is one small piece of total 
student achievement. Provincial test results and implications are so complex that no single 
factor, such as AISI, can take the credit or blame for changes. 



Recommendations 

CASS offers the following recommendations for consideration in planning Cycle 4 and 
future cycles of AISI. 

1. Cycle Length. Allow more time for projects as change may not occur in three years. 
Research on systematic and cultural change supports this recommendation. 

2. Planning Cycle. Integrate AISI into the planning cycle so there is awareness of 
announcements. This recommendation would be easily resolved with ongoing AISI 
funding guarantees so cycles were aligned and planning could be coordinated. 
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3. Innovation. Move back to the innovative risk taking of AISI beginnings. A revisit of 
the original criteria seems appropriate. Possibly an assessment of AISI innovation/risk 
taking would alleviate the perception that AISI is becoming too institutionalized within 
the department. 

4. Evaluation. Take time for a comprehensive evaluation if that is a recommendation 
from the colloquium. CASS would support and assist if it is determined that an 
objective evaluation is recommended. Our caution is that many AISI accomplishments 
cannot be measured quantitatively and, in fact, shouldn’t be. The qualitative 
improvements in teaching and schools cannot be measured with exam results. 



Paulette Hanna is President of the College of Alberta School Superintendents. She can be reached at 
phanna@rdcrd.ab.ca . 
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Chapter 7 

Enhancing Professional Practice and Mobilizing Knowledge 

Philip McRae 

University Faculties of Education 



Introduction 

The Universities of Alberta 32 , Calgary, and Lethbridge have supported the Alberta 
Initiative for School Improvement as an AISI Education Partner over the past three cycles 
(2000-2009). As AISI partners, the Faculties of Education have generated literature 
synopses and provincial research reviews of projects that have positively impacted 
student learning. The university partners have connected subject-specific expertise in the 
faculties, and worked with teachers and districts to support their efforts in capturing the 
complex nature and successes of their AISI projects. They have assisted with AISI 
project research methods and reporting, and have been challenged by problems and issues 
that have emerged during the research cycles. The universities have supported the links 
between preservice teacher field placements and AISI projects, and worked closely with 
graduate students as they conduct their research to improve their school settings. The 
university partners, over the three AISI cycles, have reviewed many of the annual reports, 
facilitated multiple focus groups, and have engaged in the work of systemic school 
improvement across the province of Alberta. 

This chapter shares some of the key findings, based on the seven provincial research 
reviews conducted by faculty from the Universities of Alberta, Calgary, and Lethbridge. 
In reporting these findings, comments are made on the implications of a large scale 
improvement initiative on a provincial learning system. These research reviews were 
conducted to inform Alberta Kindergarten to Grade 12 school jurisdictions and Alberta 
Education in their future efforts to enhance school improvement efforts. The reports and 
executive summaries can be found online at the AISI website 33 . 

The key findings in the AISI provincial research reviews (examining the work of AISI 
Cycles 1 to 3) are based on the perspectives of many Alberta communities, teachers, and 
students who have been involved in innovative work that supports student learning. The 



32 The University of Alberta includes Campus St. Jean. 

33 http://education.alberta.ca/admin/aisi/aisidocs/what.aspx. 
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perspective is that of a province-wide program involving hundreds of schools committed 
to improving student learning, while enhancing engagement and performance through 
innovative site-based research projects. 

To conceptualize AISI as one linear homogenous entity is far too simplistic. AISI is a 
complex adaptive system that is constantly in flux and changing based on the unique (and 
compelling) needs of classrooms, schools, and jurisdictions where AISI flourishes. AISI 
has supported the notion that teachers, students, and local communities have the best 
insight into what needs to be improved within their own places of learning, and that they 
can be the forces of change in those sites through a process of site-based research and 
innovation. 



Research Reviews 

Researchers at each university (Universities of Alberta, Calgary, and Lethbridge) 
conducted double and/or triple-blind reviews of AISI Project Annual Reports on 20 to 35 
projects for various themes (e.g.. Differentiated Instruction, Language Arts/Literacy, 
Mathematics/Numeracy, Learning and Technology, Collaborative Professional 
Development, and School Climate and Character Education). Alberta Education uses 
effect size analysis to determine projects that have had a positive impact on student 
learning. For themes with few projects 34 (High School Completion, and First Nations, 
Metis and Inuit), the provincial research review included all of the projects in the cycles. 
The review on Parent/Community Involvement 35 followed a different strategy. In cases 
of a small number of projects, data represent the entire corpus of projects within that 
thematic area. 

Researchers at each university then facilitated separate focus groups for each theme, with 
representation from school districts, charter, and private schools identified within the data 
set. In order to triangulate the findings, researchers then conducted a series of telephone 
interviews with additional projects that were not represented at the focus groups, but that 
were in the AISI project data set under review. In many cases the handouts used at the 
focus groups were used to structure the telephone interviews. 

The research reviews were exploratory in nature and conducted to better understand the 
issues and views on various dimensions of school improvement in Alberta; they were not 
designed to be predictive by nature. The research teams recognize the diversity and 
variance in measures and data gathering methods represented by the AISI projects under 
review for these reports. In some cases this created unique challenges with the original 
data source used within the review of site-based research, while in other cases there was a 
clear indication of research design and method. The intent of the research reviews is to 
support larger system-level knowledge mobilization. 



34 High school completion included 18 projects, and FNMI 16 projects. 

35 In this case, all APAR sections on parental involvement were included; 24 projects were selected for 
focus group participation. 
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As noted in Celebrating School Improvement 36 (Parsons, McRae & Taylor, 2006), site- 
based research conducted by many of the school authorities in the projects reviewed, 
contributed to their own contextualized understanding of school improvement through a 
reflective and iterative process of research and inquiry. This has led each particular 
community of learners towards ‘innovative’ instructional strategies, educational 
practices, and a deeper knowledge of the environments within which they support student 
learning. It should be recognized that the nature of the innovation is very specific to new 
learning or educational practices for the community undertaking the AISI project(s), and 
therefore might not be recognized as innovative in another context. 



Key Findings Across the Research Reviews 

The following key characteristics and challenges of AISI were shared at the AISI 
Colloquium based on the research findings. Each research review contains very detailed 
strategies, descriptions, and considerations specific to the themes, while the following is a 
set of shared findings from all the reviews. 



Common Characteristics 

In many cases, the research teams found the following characteristics common to the 

reviewed projects: 

• Regularly scheduled, job-embedded time for teacher collaboration was deemed 
necessary. 

• A common culture of action research and inquiry emerged from the work. 

• Significant leadership support both at school levels and at the district level was 
critical to success. 

• Empowerment and ongoing collaborative professional development for teachers 
helped to ensure that changes were sustained. 



Common Challenges 

Although each research review identified unique challenges, the following were generally 
common themes participants identified in the focus group and follow-up interviews. 

These common challenges related to achievement of project goals and to long-term 
sustainability of changes in practice. 

Staff Changes. Leadership change brings with it changes in focus and/or values, thus 
making it difficult to keep the original vision and commitment. Changes in staff, partway 
through a project, can bring resistance to change on the part of new team members, while 
taking away the developed strengths and contributing roles of departing team members. 



36 Parsons, J., McRae, P., & Taylor, L. (2006). Celebrating school improvement: Six lessons learned from 
Alberta ’s AISI projects (2 nd ed.). Edmonton, AB: School Improvement Press. 
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Limitations of Time. The importance of ongoing, job-embedded professional 
development for teachers, where they meet and share and work collaboratively, requires 
significant time that teachers want - but not at the cost of over-reliance on substitutes. 

Measuring Student Success. The selection of appropriate data instruments and the 
analysis of outcomes were significant challenges to many projects. 

Too Many Concurrent Activities. The challenges resulting from ongoing change and 
renewal, coupled with other demands on the school system (i.e., new curriculum, 
technology pilots, etc.), resulted in fatigue that was an issue for some initiatives, 
particularly if the AISI project was not in alignment with school and district goals. 

Meaningful Parent Involvement. Projects identified involvement and engagement of 
parents, beyond newsletters and meeting with the school council, as a significant 
challenge. 

Sustaining Changes in Practice. As one teacher put it, “Real change takes time; it 
doesn’t happen in three years.” The challenge is that of sustainability when the AISI 
project cycle finishes and resources are directed elsewhere. 



Conclusion 

The consistency of themes and issues represented were significant among written AISI 
Project Annual Reports, between written reports and findings from the many focus 
groups and telephone interviews, and in some cases between report findings and the 
research literature. Such consistency is encouraging, and is indicative that AISI is indeed 
making a positive contribution to the goal of improving student learning through locally 
developed and implemented projects that address unique needs and circumstances. 

At the AISI Colloquium the following considerations were put forward with respect to 
the future of AISI. 

1. How might school jurisdictions encourage and preserve a diversity of school 
improvement projects for each cycle? The harmony between balancing many small 
projects, and larger all inclusive projects, is a delicate one that should carefully 
consider how to better encourage the vigorous and free exchange of ideas, and 
preserve the diversity of projects within the system. 

2. To what extent can AISI projects engage the entire school community in enhancing 
school improvement? Perhaps through a sense of distributed and contextual 
leadership the participation in AISI will continue to broaden itself into the wider 
community. 
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3. What are the essential questions and research measures that will best inform the 
unique needs and circumstances of the A1SI projects? Change in school culture, 
longitudinal impacts on professional practice, and the measurement of student 
learning are complex processes, and school jurisdictions should continue to struggle 
with the most efficacious ways of measuring school improvement. There is not one 
simple approach or answer when struggling with very complex education issues. 

4. How can A1SI projects encourage a sense of risk taking and innovation in the 
community of learners that leads to positive change? As school communities engage 
in their site-based research to improving student learning, collective inquiry has the 
potential to create fundamental shifts in thinking, and the risk taking and innovative 
actions that emerge over time have the potential to evolve into continuous growth and 
system level school improvement. 

The collective wisdom emerging from the research reviews can inform Alberta schools, and 
national and international education communities. By exploring the key findings, we also 
hope that some of the identified challenges facing our school communities are recognized. 

As large scale school improvement initiatives become more prevalent where provinces/states, 
school districts, schools, and classrooms seek to improve student learning, AISI has the 
potential to become an even more significant network through its many localized initiatives. 



Philip McRae is the AISI Director, Faculty of Education, University of Alberta. He can be reached at 
pmcrae @ ualberta.ca . 
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Part 2 



School Authority Perspectives 




Chapter 8 

AISI - Winds of Change: Shifting to Collaboration 

Ian Osborne and Sheldon Rowe 
Peace Wapiti School Division 



Peace Wapiti covers 23,000 square kilometers and serves approximately 5,500 students 
in 27 schools with a teaching staff of 330 full-time equivalent teachers. The division 
operates under certain limitations including a large geographic area, small schools and 
multi-graded classrooms; some communities are experiencing rapid and transient growth, 
while others are quite stable but declining in population, and an aging workforce. 



Impact of AISI on Peace Wapiti School Division 

Every AISI cycle has been a valuable learning experience. Projects in Cycles 1 and 2 
were not generally sustained because they depended on increased staffing to fulfill their 
objectives. Once funding ended, staffing was reduced to pre-project status thus 
terminating the benefits. This experience gave birth to the structure of Cycle 3 projects 
which promise greater hope in having a long-term positive impact on student learning, 
effectiveness, and sustainability. 



Cycle 1 

Cycle 1 projects, of which there were initially 11, but reduced to 10 at the end of the first 
year, consisted of the following areas: curriculum implementation, reduction of student / 
teacher ratios, and special education. Of these projects, four had a system-wide focus: 
ICT Curriculum Support, Curriculum Implementation Support Enhancement, Increased 
Counseling, and Extended Early Literacy. The remaining six projects were in individual 
schools. 

The process used in the identification and selection of these projects was ambitious. 
There was a significant degree of stakeholder input. What became apparent, in 
retrospect, is that if the parameters are not set, stakeholder input becomes personalized 
with the various publics advocating for their own desires and not what may be best for a 
school, or a division. 

Cycle lprovided clear evidence that with a focus on increased staffing: 

• Gains are short term and last only for the duration of the project. 
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• Benefits are unsustainable - once the finances ended so too did the benefits. 

• A scatter gun approach - giving schools full autonomy, without setting the structure 
and focus - does not move the division in a common and desired direction for the 
benefit of all its students. 

• Without a planned and coordinated effort, professional development improvements 
were minimal. 



Cycle 2 

Cycle 2 narrowed the focus to three projects. Principals proposed projects based on their 
individual school’s needs and took these proposals to their school councils. The School 
Board selected three from the list of proposals. The three projects were Students at Risk 
(STAR involving 4 schools), Special Education Mathematics (12 schools), and School 
Climate (8 schools). 

As in Cycle 1, there was no system- wide focus. STAR and Special Education Mathematics 
relied on increased staffing and were geared to a small percentage of the division’s student 
body. Some schools did capitalize on the School Climate Project and incorporated a 
professional development component, while others seized the opportunity to increase 
administrative time and incorporated some aspect of addressing the issue. 

It was during Cycle 2 that the division embarked on a pilot project of establishing Professional 
Learning Communities (PLCs) with ten schools. Teachers were given school time to 
collaboratively engage in improving/enhancing some aspect of their school. The success of 
this event morphed into a system-wide adoption of this structure. It was at this time that 
schools involved in School Climate incorporated the model of PLCs into their operations. 

Cycle 2, in conjunction with the PLC initiative, revealed the following educational secrets: 

• Teachers, when given the structure for focused professional development, will engage 
in meaningful and productive dialogue. 

• Teachers will work towards specific achievable goals within a time frame and be 
successful in their endeavors. 

• Administrative commitment is a key determinant in the success of a professional 
learning community. 

• Teachers prefer job-embedded time to engage in professional learning activities as 
opposed to time after school and/or on weekends. 

Towards the end of Cycle 2, three key questions arose. Addressing these questions led to 
the structure - believed to be very successful - of the Cycle 3 projects. 

1. What if a professional learning community model were used in Cycle 3? 

2. What if key teachers in certain disciplines (now called Curriculum Coordinators) 
were identified and appointed to coordinate, plan, manage, and lead system- wide 
professional development endeavors within this structure? 
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3. And what if schools were to appoint key teachers (now called Lead Teachers) to be 
active participants and learners to work with the Curriculum Coordinators to research 
and incorporate proven best practices into their teaching repertoire? 



What Areas Did Cycle 3 Projects Focus On and Why? 

Cycle 3 saw a dramatic shift in philosophy on how AISI projects were viewed by the 
system. Driven by innovation (PLCs), Alberta Education’s Accountability Pillar, a desire 
to attract as many teachers as possible, and to positively affect a large percentage of 
students, the division embarked upon four projects: 

• Literacy - Divisions 37 1 to III, unofficially expanded to incorporate Division 4 
teachers 

• Numeracy - Divisions I to IV 

• Social Studies - Division IV 

• Assistive Technology for Learning - to assist students whose scholastic progress is 
being hampered by their reading challenges 

Zone 1 schools have perennial issues with Provincial Achievement Tests and Diploma 
Examinations and so too do schools in this division. System administrators, school 
administrators, and school councils all agree that a coordinated effort between both 
individual schools and a system-wide PLC offers a powerful and hopefully effective 
solution to solving the issue of performance on provincial tests. 

Whereas to some PLCs may not be innovative, to this division innovation lies in the 
following: 

• A system-wide PLC spearheaded by nine curriculum coordinators in four different 
disciplines. 

• Subdividing each discipline into levels or divisions I through IV. 

• Attempting to have over 100 teachers, roughly 15 - 20 per division, moving in one 
direction of professional development activities as decided upon by the division is an 
ambitious enterprise. Teachers were involved in collaborating, sharing ideas, and 
learning new teaching strategies to enhance their delivery of instruction for purposes 
of improving student achievement and learning, and meeting anywhere from four to 
six times per year. 

• Introducing, developing, and utilizing the moodle platform, while inservicing teachers 
on its usage. The Moodle Site 38 is an ‘e-supermarket’ of education information where 
teachers, for example, may share strategies, post tests, and lesson plans, have e-links 
to educational sites, and collaborate online between face-to-face meetings. 



37 Division I includes Kindergarten to grade 3, Division II (grades 4 to 6), Division III (grades 7 to 9), and 
Division IV (grades 10-12). 

38 http://moodle.org/ . Moodle is a free, open source course management system for online learning. 
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Reflecting on Cycle 3 to date, we have learned: 

1. In year one, teachers gathered with their own unique styles, prejudices, and egos to: 

• Break down barriers 

• Create trust 

• Begin to collaborate and map the course of professional development activities 

• Collect data and adopt best practices for improving student learning 

2. In year two, teachers continued to learn and assimilate new ideas having overcome, 
essentially, those barriers that may have impeded progress. 

3. In year three, teachers are attempting to shift the focus to classroom implementation 
of new ideas for student learning, while continuing the process of learning and 
assimilating identified ideas and strategies. There is also a concerted effort to create a 
stronger link between the schools’ PLCs and the system- wide PLCs. 



Lessons in Meeting the Learning Needs of Students 

There is no clear-cut evidence that professional development translates into improved 
learning for students. However, if “insanity is doing the same thing and expecting different 
results”, it stands to reason that if there is a need to improve student achievement, teachers 
must change the way in which lessons are delivered. Therefore, we believe that by 
exposing teachers to research-based effective teaching strategies, and allowing them the 
opportunity to digest, assimilate, and internalize emergent information, one could expect a 
positive impact on student learning. We believe the present structure of AISI projects, and 
the influx of new teaching strategies, should lead to improved student learning. 

In the quest for meeting the learning needs of students, it stands to reason that the 
learning needs of teachers must first be met. The bedrock upon which this division hopes 
to accomplish this requires the following: 

• Development of a strong Professional Learning Community that supports and 
encourages investigation, adoption, and sharing of best practices. 

• Strategically placed working meetings where teachers are provided the opportunity to 
collaborate and build community. 

• Effective utilization of the moodle site, which is being used as an extension of the 
working meetings. 

The journey so far has unearthed four major lessons. 

1. A culture of collaboration takes time. 

• The first year was spent breaking down barriers of trust that hampered the 
collaborative process. Trust issues among and between teachers, coordinators, 
and administrators prevailed and were overcome, to a large degree. The system- 
wide professional development activities began. 
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• Year two saw the growth of the collaboration process as new leanings continued. 

It also heralded cycles of presentations, discussions, implementation, reflection, 
and review. 

• Year three is continuing the presentations, discussion, implementation, reflection, 
and review with a concerted effort to connect new teaching strategies to student 
learning and achievement. 

2. Work needs to start with understanding the curriculum. 

• Many of our teachers are required to teach outside their area of expertise. Others 
have not examined or reviewed the curriculum in quite some time. Both of these 
types of teachers need and benefit from ongoing professional development to 
assist them in not only knowing what to teach but also what strategies are best 
suited for the delivery of instruction. 

• The above comment was very apparent in our Literacy Project. A significant 
amount of time was spent in translating the curriculum into ‘I can’ statements. The 
process was most valuable to participating teachers and ensured that they became 
familiar with the curriculum. They were then able to engage in meaningful 
dialogue about which strategies were best suited to achieving a specific goal. 

3. To meet student learning needs there is a need to focus on results. 

• The first and perpetual challenge is to find the best measures to test student 
learning. Cycle 3 is using a combination of the following: Provincial 
Achievement Tests, Diploma Examinations, Canadian Tests of Basic Skills, 
Canadian Achievement Tests, and locally made common tests. 

• Use of a myriad of tests has raised questions of over testing. At what point does 
the assessment of learning become redundant and burdensome? It may be better 
for Peace Wapiti to embark upon a standardized testing and evaluation program in 
which all schools use the same agreed-upon tests. 

4. Professional conversations inspire teacher efforts to improve student learning. 

• Teachers, who work in rural settings in small multi-graded schools, work in 
isolation, essentially having no one with whom to engage in professional 
discourse on the subject area and grade taught. 

• These teachers instruct a wide range of subjects and, of necessity, need to 
implement a variety of instructional strategies. 

• The professional conversations and collaboration offered by a PLC can enhance 
and build teachers’ instructional effectiveness. 

As we continue to learn from and develop these revelations and accept their counsel, the 

probability of enhanced student achievement and learning increases. 
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AISI Enhancements 



District Level 

It has taken several years to fully realize the power of AISI projects. System administrators, 
school administrators, curriculum coordinators, and lead teachers will continue to work 
together to ameliorate the achievement issues as identified by the Accountability Pillar. 
Certain areas have already been identified to take us closer to our objective - improved 
student learning and achievement. These are: 

• Increased collaboration between AISI coordinators and school administrators to 
synchronize school-based PLCs to the system-wide PLCs. In this way, a greater 
number of teachers would be affected and the cascading process would become 
effective. 

• Appropriate student learning measures should be decided and adopted system-wide 
and the analysis of results used to shape learning. 

• There should be planned and coordinated articulation between divisions. This would 
increase the probability that teachers are focusing on areas that are essential and that 
students are better prepared for success when promoted to another division ( for 
example from grade 3 to grade 4). 

• With a large percentage of the teaching force facing retirement, a proper orientation 
plan, in keeping with the AISI philosophy, for new teachers, needs to be in place. 



Provincial Level 

The following recommendations are offered in the spirit of enhancing the AISI program. 

AISI leaders have done and are continuing to provide excellent services and leadership. 

• One major concern has been addressed. In past cycles, the late announcement of AISI 
continuation left little room for in-depth planning. With the early announcement of 
Cycle 4, this impediment no longer exists. 

• Although the universities have been valuable partners, those of us who teach in the 
north are separated by quite a distance from our nearest university partner. How 
about considering a college (or campus) situated in the ‘real’ north for partnership or 
associated partnership? Closer proximity might increase the efficacy of research 
study, adoption, and implementation of teaching strategies. 

• AISI conferences, workshops, and symposia provide intermittent opportunity for 
province-wide networking. How about considering a concerted effort towards the 
creation of a sustainable provincial AISI networking system? 
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Cycle Four and Beyond 

We have begun planning for AISI Cycle 4. At the first biannual General School Council 
meeting this year, AISI’s purposes and intent were explained to participants. Parameters 
and structures were outlined and school councils were provided with the necessary 
information for constructive and informed input into the new cycle of AISI. 

We believe that our present structure - curriculum coordinators, working meetings with 
lead teachers, coordinated professional development initiatives, collaboration, utilization 
of the moodle site, and building professional learning communities - provides a blueprint 
for future projects. 

It is anticipated that there might/should be a change in language used - from professional 
development for teaching practice to professional development for student learning. 

There will be continued efforts to strengthen the relationship between the schools’ PLCs 
and that of the division. 



Final Thoughts 

The school improvement process has been long, tortuous, sometimes arduous, occasionally 
frustrating, but always rewarding! AISI has provided an innovative vehicle to address, 
enhance, and improve teacher performance with the ultimate goal of improving student 
learning and achievement. It has created a platform for ‘outside the box’ thinking, gently 
nudged many teachers away from their comfort zone, and initiated the critical shift towards 
engaging students effectively in the learning process. The unifying structure of professional 
learning communities, established during Cycle 3, effectively sets the context for continued 
collaboration and effective, cohesive planning for improved student learning in Cycle 4. 



Ian Osborne is AISI Coordinator for the Peace Wapiti School Division. Sheldon Rowe is Superintendent. 
They can be reached at ianosbome@pwsd76.ab.ca and sheldonrowe@pwsd76.ab.ca . 
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Chapter 9 

AISI: Growing From Good to Great 

Jenise Bidulock, Corrie Ziegler, and Debbie Jackson 
Edmonton Public Schools 



Introduction 

There are no perfect people. There are no perfect projects. We are not measured against 
perfection, only called to do what we can, to set out on an exploration to an imagined 
destination, an imagined good. (Westley, Zimmerman, & Patton, 2006, p. 229) 

Edmonton Public Schools set out on an exploration to enhance student achievement and 
high school completion rates through their work related to the Alberta Initiative for 
School Improvement (AISI). What began as “school” improvement nine years ago has 
grown to become “system” improvement as staffs have worked collaboratively to shift 
district culture. What the district has learned over the past three AISI cycles will assist it 
in growing this initiative from something good to something great for all stakeholders in 
the service of student learning. 

Cycle 1 of the district’s AISI work included many projects created by individual schools 
responding to the basic question underpinning AISI - “What would you do to improve 
your own school?” Through this cycle schools learned valuable lessons about improving 
teaching and learning in their individual school situation. Great ideas were born but 
those great ideas did not take root in other locations because there were few vehicles for 
sharing successes and there was limited collaboration at the system level. Many schools 
put their funding into providing additional staff to support programs such as Reading 
Recovery and small class size. This may lead to improved student results but does little 
to engender system-wide change. Without a means to share the success of these 
programs, system improvement can not be realized. Finally, this use of AISI funds is 
often not sustainable. 

Building on the lessons and achievements of Cycle 1, Edmonton Public Schools (EPS) 
embarked upon Cycle 2 with an eye to improving student achievement through system- 
wide supports and change. A district-wide project called Focus on Supporting Teaching 
and Learning (AISI Project 10023) was implemented. This project was an ambitious one 
that involved school-based leaders from every school working together to build a culture 
of continuous improvement. Tremendous growth was seen in the area of collaboration. 
There is now increased collaboration within schools and across schools; increased 
understanding of the use of data to inform practice; and the role of the principal has 
shifted toward instructional leadership. As with all efforts to promote great change there 
was also great challenge and great success. 
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One additional success from Cycle 2 was a small project involving 18 high needs schools 
that began a full-day Kindergarten program. The success of this project has led to the 
district funding full-day Kindergarten in 25 high needs schools. 

Lessons from the Framework for Supporting Teaching and Learning provided a 
foundation and were incorporated into the planning for Cycle 3 AISI work. Cycle 3 has 
been designed to build on the culture of collaboration and the emphasis on instructional 
leadership by the principal that included a leadership team within each school. Again at 
the heart of all work was a focus on student learning. 

District schools are currently participating in one of five AISI projects in Cycle 3. These 
projects are rooted in the successes of previous cycles and have built in choice for 
schools. All projects are improving teaching and learning through collaboration and 
broad-based leadership with district-wide supports. The goal of all five projects is to 
provide enhanced supports for students at-risk so that more achieve grade level and more 
students complete high school. 

Through a process of extensive consultation involving school and central staff including 
senior management, the five projects were developed. School leaders worked together 
with central leaders to develop plans and targets for the projects. Schools expressed a 
desire to meet their individual needs and so choice became a priority. Schools were 
given choice in terms of which project they would participate in for Cycle 3 based on the 
learning needs in their school. 

EPS use a distributed leadership model. Project Leadership Teams (PLTs) lead each of 
the projects. These teams are composed of school-based personnel (teachers and 
administrators) as well as central-based personnel. Input from schools in the projects, 
research, and collective wisdom are all used to make decisions about project work. 

Staff in Edmonton Public Schools subscribe to Michael Fullan’s belief that organizations 
need to learn continuously and to shift intelligently as they work to bring about deep and 
lasting change (Fullan, 2008a). As the district builds toward Cycle 4 in AISI and beyond, it 
believes it is important to continue to carry forward the important lessons learned, to build 
on successes. In this way the district has the opportunity to grow from good to great. 

How does a district grow from good to great? In the words of Jim Collins, it must 
“. . .discover a simple, yet deeply insightful, concept for making really big choices and 
focus on the few things that will have the greatest impact” (Collins, 2001, p. 87). In EPS, 
the question for making the really big choices is always: Is this in the service of student 

learning? 
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What is “Good”? 



Over the past three cycles, some key ideas and strategies have had the greatest impact in 
meeting the goal of enhanced student learning. They include building and distributing 
leadership, fostering a culture of collaboration, shifting the focus from teaching to a focus 
on learning, and providing the opportunity for choice. 



Building and Distributing Leadership 

Leaders matter. Leaders’ thoughts and actions shape the culture of their organizations 
and set the direction and pace for the professional learning and teamwork that are 
essential in improving organizational performance. (Sparks, 2007, p. xii) 

Leadership capacity in EPS has grown immensely over the past three cycles. Teacher 
leaders and leadership teams are in every school site. Teacher leaders support walk- 
throughs and classroom visits; teacher leadership teams run meetings and professional 
learning opportunities such as looking at student work and analyzing data. Leadership 
teams have learned skills to coach effectively, facilitate meetings, and organize 
professional learning for staff. Project Leadership Teams orchestrate AISI project work 
and work together effectively. The definition of leadership has broadened. Building 
capacity has been purposeful and incremental. 

In Cycle 2 building an Instructional Leadership Team (ILT) at each school and a focus on 
the principal as instructional leader were highlighted. Principals were supported in their 
work first by coaches and then as coaches, with training and professional development. 
Principals and ILTs came together to build collaborative frameworks at district-wide days 
and then more deeply at their schools. 

In Cycle 3 principals and school-based leaders work together to support the work in 
schools. School and central staff provide leadership through PLT participation and are 
supported in further developing their leadership capacity through regular conversation 
and work with Dennis Sparks via videoconferencing. This deeper understanding of 
effective leadership continues to be refined and redefined over time. 

Another critical leadership role has developed over this cycle and that is the AISI 
coordinator role. Coordinators work hand-in-hand with the PLT to create mutual 
accountability, to ensure that AISI work meets criteria and builds on research. The 
coordinator manages the details producing an environment that fosters creative thinking. 

Capacity building in EPS has taken the form suggested by Fullan: start by attracting 
talented people and then help them continually develop individually and collectively on 
the job (Fullan, 2008a, p. 63). 
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Fostering a Culture of Collaboration 

Linked closely to shared leadership and capacity building is collaboration. The old 
saying ‘many hands make light work’ is truer than ever in today’s classrooms and 
schools. By learning and sharing solid teaching practices side-by-side with effective 
leaders the right work gets done. Well-functioning groups are always better at addressing 
challenging task (Fullan, 2008a, p. 52). In Cycle 2, schools followed tight protocols for 
Instructional Walk Throughs. This built the foundation for today’s work in schools that 
continues to see teacher teams and individuals in classrooms learning from each other 
and informing practice. Cohort work in the past was highly supported through external 
expectations and largely consisted of three schools working together to build toward a 
systemic culture of collaboration. Today cohorts continue to operate but flexibility of 
process exists to meet school needs. 

Collaboration is increasingly becoming the way district staff works. Today in schools, 
staff is shifting from professional development to professional learning. Schools are 
working with consultants and embedded leaders to look more closely at student and 
teacher work as the way to improve. Coaching is beginning to expand so that now 
teachers are involved in being coaches and being coached by another teacher. 

Collaboration is supported through networking that occurs at times such as May Sharing 
- an annual event in which schools share their successes with other schools in the project 
in relation to their school’s AISI work. Schools are also given time to network at 
professional development sessions. Some sharing occurs across projects through school 
visits and the sharing of school stories. Other vehicles for collaboration and sharing 
include meetings with senior administration; presentations to the public and the Board of 
Trustees at the Public Board Meetings; conversations with other jurisdictions through 
Teachers Convention; provincial, national, and international conference presentations; 
and articles in professional and educational publications. In Edmonton Public Schools 
staff is beginning to say, “Educating OUR children is all of OUR work”. 



Providing Choice 

One other over-arching theme within the district is that of choice. At EPS staff is proud 
to be a district of choice for parents and students. There is pride in the history of school- 
based decision making that meets the needs of the communities of learners. Choice must 
be honored. Adult learners come with diverse interests, backgrounds, and needs, and 
their diverse needs must also be accommodated. AISI work has provided support to 
schools in choosing their focus through project choice. Schools also choose their cohort 
schools and individuals choose with whom they coach and the coaching model. Finally, 
schools choose the individuals who serve as leaders in their school. This choice has been 
supported and built toward in purposeful and thoughtful ways that support the broad 
goals relating to improved learning for students and improved high school completion 
results. 
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Growing to Great: Cycle 4 and Beyond 

As the district moves toward Cycle 4 and beyond it is important to continue the momentum 
from good to great. EPS must identify and support the district-wide practices that are too 
important to leave to chance and address the challenges. The district must take to heart 
Michael Fullan’s observation that it, like many school districts, suffers from ‘initiativitis’, 
(Fullan, 2008b, p.l). Too many projects, too much professional development, and too 
much fragmentation lead to lack of coherence, success, and change. 

As the district begins to envision where it must go next, the following themes are 
emerging: close the knowing/doing gap (Pfeffer & Sutton, 2000), deepen support for 
instructional leadership, practice wise and continuous use of data to inform practice, and 
build coherence. These are the next steps to sustainability. 



Close the Knowing/Doing Gap 

District staff must take up the challenge of making learning the work. This will involve 
moving toward professional, job-embedded learning where all teachers and all school 
staff work collaboratively toward continuous improvement. Most educators know more 
about effective practice than they regularly employ in their work. Consistent use of what 
is already known would lead to significant improvement in leadership, teaching, and 
learning (Sparks, 2007, p. 179). As a district, staff has spent much time learning about 
best practices, but now they need time to apply, practice, refine, and implement what they 
have learned at professional development sessions. 

Time built INTO the day is necessary for staff to do this work. Time to look closely at 
lesson and unit plans, assessment practices, how to differentiate to meet student needs, to 
deepen literacy practices, and to integrate technology. According to Fullan (2007), 
improvement is a function of learning to do the right things in the setting where you work. 
Teachers should be encouraged to engage in continuous and sustained learning about their 
practice in the settings in which they actually work, observing and being observed by their 
colleagues in their own classrooms and classrooms of other teachers in other schools 
confronting similar problems (p. 35). In this way there will be greater engagement of the 
hearts and minds of all teachers, and increased capacity to meet the complex learning needs 
of individuals, schools, and the organization. Professional development workshops simply 
do not have an impact that lasts. A 1995 study by Joyce and Showers illustrates the power 
of job-embedded learning: when theory, demonstration, practice, feedback, and coaching 
are combined the transfer of new learning to the classroom is significant. It is through a 
cycle of the application of theory, the use of ongoing demonstration and practice that is 
supported by feedback and coaching that lasting change in teaching practice occurs (Killion 
& Harrison, 2006, p. 15). 
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Deepen Support for Instructional Leadership 

Leadership is second only to classroom instruction among all factors that contribute to 

what students learn in school. (Leithwood, Louis, Anderson, & Wahlstrom, 2004, p. 3) 

Principals engage in very complex work every day. Their role as instructional leaders is 
critical and cannot be supplanted by management issues. In their role as instructional 
leader, principals become a critical part of the change process. Principals must have 
capacity to coach, to commit to and communicate vision, to support staff through change, 
to learn in context and to support school AND district change simultaneously. To do this 
work principals must also have time to learn, to reflect, to collaborate, and to work with a 
coaching partner toward transformation. Principals must take on the role of building 
coherence across the district by supporting staff in these changing times. They must help 
staff examine practices and lead the learning in their school. 



Practice Wise and Continuous Use of Data 

As the district moves into Cycle 4, it has learned the value of careful data analysis to 
inform next steps. Utilizing more precise data tools that measure what needs to be 
measured will be explored. Triangulation of data is important; student, parent, and 
teacher voices have much to offer as a way to interpret and inform the more traditional 
types of data. Growing to greatness means that at the district level there must also be a 
continuous practice of wise and continuous use of data. 

In the past, data were not always shared with all stakeholders. There has been a shift 
towards more transparent use of data. As a result, teachers, principals, parents, and 
central staff are now demonstrating a deeper hunger for data from a variety of sources to 
celebrate success and to inform next steps in instruction in the service of student learning. 



Build Coherence 

Building coherence across schools, with parents and our community, is necessary to grow 
from good to great. All stakeholders need to understand how each action, and all actions, 
are in the service of student learning. It can be overwhelming, especially for teachers in 
the classroom. This can lead to statements such as, “This too shall pass”. The goals of 
education, district priorities, school plans, and classroom plans all provide guidance for 
the important work of teaching and learning. AISI supports this work. It is not an “add 
on”. It is important to provide meaningful opportunities for dialogue and reflection so 
that the connections become obvious, leading to sustainability of practice. Helping 
stakeholders connect the dots is critical. The dot connection is quite simple - in the 
service of student learning. 
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Summary 



This chapter begins and ends with the same quotation: “There are no perfect people. 
There are no perfect projects. We are not measured against perfection, only called to do 
what we can, to set out on an exploration to an imagined destination, an imagined good.” 
Over the past three cycles, Edmonton Public Schools has cultivated and nurtured 
continuous improvement through its AISI work. Many good things have happened as a 
result. From the good will come something great in the service of student learning. The 
quotation goes on to say: “So forget about the fear, forget about the guilt, forget about 
the fact that the doorway makes no promises, just step through” (Westley, Zimmerman & 
Patton, 2006 p. 229). AISI is all about “stepping through”, taking the risk in the services 
of student learning. 
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Chapter 10 

AISI Footprints in Pembina Hills 

Judy Lefebvre and Richard Harvey 
Pembina Hills Regional Division 



Footprints are the impressions or images left behind by a person walking. In Pembina 
Hills, AISI footprints are readily seen, and they continue to become more visible, as the 
division engages in furthering its actions to meet the AISI goal of improving student 
learning and performance by meeting the unique needs and circumstances in its schools 
and classrooms. In the past nine years, there have been many changes experienced in this 
division, and the following account represents the influence of AISI projects on that 
growth and development. 



Teacher Professional Development 

Joellen Killion (2005) has stated that students are not likely to perform at higher levels 
until teachers begin performing at higher levels. The teacher professional development 
experience in Pembina Hills gives credence to Killion’ s statement with powerful growth 
in teacher learning accompanied by improved student learning, which began with the 
onset of AISI, and continues today. 

In Pembina Hills, the opportunities created by the AISI projects have given license to 
research which has translated into practice and has resulted in improved student 
achievement and a collaborative culture among staff. 

Prior to the advent of AISI, teachers in Pembina Hills were on their own - individual, 
sometimes isolated - course of development in their professional growth. Experiences 
varied widely as each sought to fill in gaps in effectively implementing curriculum for the 
students in their charge. As the possibilities for improving the professional learning for 
teachers was researched, it was discovered that there needed to be a comprehensive, 
sustained, and intensive focus on professional development for it to garner the results 
hoped for. Changes in teacher practice, opportunities for collaboration, and classroom- 
embedded professional development were phrases that frequently recurred in the 
research. 

The first efforts to achieve this professional development goal were in Early Literacy, a 
focus of AISI Cycle 1 projects. Extensive research was conducted, and armed with the 
knowledge of what courses of action in professional development were providing the best 
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results, a team consisting of classroom teachers, school administrators, and the Director 
of Instructional Services embarked on a province- wide tour of schools that had gained the 
reputation of providing superior programs in early literacy. 

The consistent factors discovered were completely aligned with the research: collaboration, 
classroom-embedded professional development, and a direct focus on student improvement 
were manifested in schools which were experiencing the greatest success. Most, but not all, 
of the schools observed used the Lead Teacher model, and the division soon began supporting 
the training of ‘lead’ teachers, who then took their knowledge and skill into classrooms where 
they engaged in a demo-coach model of professional development. Teachers responded very 
well to this classroom support, and soon a common language and practice developed 
throughout the division’s kindergarten to grade 3 language arts classrooms. Teachers who 
had long felt isolated were now able to share their practice, their success, and address their 
challenges with others. 

As AISI expanded its focus into upper elementary, and then junior and senior high, there 
was a parallel growth in how the division approached the professional development of 
teachers. Training of Lead Teachers in specific disciplines precipitated the onset of 
capacity building as leaders emerged at all levels. As the division was nearing the end of 
AISI Cycle 2 and planning for Cycle 3, teacher growth and development received support 
as a necessary and vital component of new projects. Embedded professional development 
was again seen as having a great impact on student learning. Research teams again began 
reviewing the best practices for teacher growth in effective schools, and the Professional 
Learning Community (PLC) became an entity in Pembina Hills. Collaboration, facilitator 
training, addressing the needs of at-risk learners, and being accountable for results became 
the essence of the PLC. Lead Teachers were appointed in each school, and resources were 
put in place enabling all teachers to engage in a PLC for seven half days throughout the 
school year. Training in facilitating a PLC was provided for Lead Teachers, and by the 
end of year one, many schools were ready to use the PLC as a vehicle for addressing 
student needs through the adoption of assessment for learning strategies in informing their 
instruction. 

During Cycle 3, the division Professional Development Committee adopted the procedure 
of supplementing the work of the PLCs in schools by devoting a portion of division PD 
days to grade-level collaborative groups with an assessment focus. This served to extend 
the work that the PLCs were doing in schools as teachers engaged in professional dialogue. 
Teachers again found an arena for support and a place to discuss specific subject areas with 
colleagues who were involved with the same curriculum. 

In preparation for AISI Cycle 4, and to further solidify the professional learning of 
teachers, a new feature of the AISI PD model was added for the final year of Cycle 3. 

Three Lead Teachers have received training as Coaches in the Rick Stiggins Classroom 
Assessment for Student Learning (CASL) process. These designated coaches provide 
leadership and support for specific schools which are arranged in Learning Teams. This 
model enables teachers to work with their coach at various levels - at-elbow in the 



AISI Colloquium 



62 



Large Scale Improvement 




classroom, during a PLC, or in a small group. The coaches facilitate the teachers’ work 
in designing assessment tools as well as aligning assessment with instruction. In this first 
year of the Learning Teams initiative, relationships are being built, planning and 
instruction are receiving support, and collaboration has been taken to a new level as 
schools have been placed with schools of a similar demographic makeup. 



Improved Student Learning 

Cycle 1 

Planning for AISI Cycle 1 involved meetings of representatives of crucial stakeholder 
groups within Pembina Hills. Needs assessments were conducted, committees were 
struck, and decisions were made relative to where the greatest effects could be realized 
through the AISI projects. The final decision clearly indicated that the greatest needs 
were in literacy and numeracy, with a predominant focus on early learners. 

The resulting projects included Extended Kindergarten for at-risk students. Early Reading 
Intervention for grades 1 and 2 students, Balanced Literacy for all K to grade 3 students, 
Literacy Across Boundaries for grades 4 to 9, and Summer Reading. Extended Kindergarten, 
Early Reading Intervention, and Summer Reading were offered as safety nets for students 
who were showing signs of not meeting expected grade level standards in literacy. All three 
projects proved to provide the necessary support to these groups of students. System Unified 
Math addressed the numeracy needs for students in grades 4 to 9. 



Results - Cycle 1 

• The Extended Kindergarten Project served the needs of 284 students, and targets were 
exceeded in all three years of the project. 

• During Early Reading Intervention, the division percentage of grades 1 and 2 students 
who were at grade level rose from 70% to 90% over a five-year period. 247 students 
were served in Cycle 1, and, of these, over 90% achieved grade-level standards. 

• Grade 3 PAT 39 reading results exceeded provincial averages in both the acceptable 
and excellence standards for two of the three years of the project, and they continue to 
do so today. 

• Literacy Across Boundaries students had 18% writing at the level of excellence in the 
third year of the project. 

• Summer Reading served 800 students - 85% maintained or improved their reading 
level. 

• System Unified Math results in the grade 9 PAT exceeded provincial averages for all 
three years of the project. 



Figure 10.1. Results of AISI Cycle 1 



39 Provincial Achievement Tests. 
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Cycle 2 



As planning for Cycle 2 evolved, it was evident that the outstanding literacy and numeracy 
results from Cycle 1 influenced discussions and decisions relative to selecting Cycle 2 
projects. 

Balanced Literacy gained a new audience in grades 4 to 6, while a Comprehension and 
Communication Project provided support for teachers in junior and senior high classrooms. 
The majority of Division II elementary language arts teachers participated in the Balanced 
Literacy initiative while 25 grades 7 to 12 teachers met to learn about strategies that would 
replace many traditional approaches to teaching reading and writing in those grades. 
Teachers also began focusing on best practices that enabled students to become successful 
learners. 

Numeracy became the focus for 56 K to grade 3 teachers, and 14 grade 7 to 10 teachers in 
the Math in Motion Project. The major objectives of this project were to improve math 
learning among students and to build a community of math teachers who would collaborate 
and offer guidance and support to each other. Trevor Calkins facilitated PD for the K to 3 
grade groups, while locally trained Lead Teachers worked in all classrooms with teachers 
and students on a regular basis. Demonstrations, modeling, and facilitating study groups 
became regular teacher experiences through this project. Saturday tutorial support was 
offered to grade 12 students in order to address specific needs and challenges of students 
who would eventually write a math diploma exam. Division tests were also created for 
grades 7 and 8. 



Results - Cycle 2 

• CTBS 40 baselines in reading were exceeded by all groups involved in the 
Comprehension and Communication Project. 

• Grade 9 language arts PAT results improved and exceeded baselines. 

• Local measures in language arts for Division II (grades 4 to 6) students met and 
exceeded standards. 

• The Grade 3 math PAT results exceeded provincial averages for all three years of the 
project. 



Figure 10.2. Results ofAISI Cycle 2 



40 Canadian Tests of Basic Skills. (1997). Toronto, ON: ITP Nelson Publishers. 
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Cycle 3 



As teachers experienced success in new literacy and numeracy practices, the growing 
awareness of the need for assessment balance and quality triggered the focus of projects 
in Cycle 3. Throughout the course of both of the previous cycles, assessment for learning 
repeatedly surfaced as an integral component of purposeful classroom instruction. 
Building on the success of the earlier AISI projects and on extensive research. Classroom 
Assessment with Purpose, and Student Engagement were adopted as the Pembina Hills 
Cycle 3 projects. Professional Learning Communities are found in every school in the 
division, and involve participation from every teacher on staff. Results of this approach 
are encouraging, although this collaborative effort involves much risk taking and a trust 
relationship in order to fully succeed. Results are not as rapidly seen in student 
improvement because the application of effective assessment practices is oftentimes very 
far removed from traditional practice. However, success is being experienced as teachers 
gain security in their new Learning Teams and look to each other for support. 



Results - Year Two, Cycle 3 

• Assessment of French Immersion students in correct oral French use improved from 
44% to 82% in Division I (grades 1 to 3) students. 

• Assessment of French Immersion students in correct oral French use improved from 
61% to 64% in Division II (grades 4 to 6) students. 

• Results in 9 out of 10 grades 3, 6 and 9 PATs improved over the baseline, ranging 
from 0.5% to 6.5% improvement in specific subject tests. 

• Locally determined measures indicate that students not writing a PAT in language 
arts have met or exceeded targets. 

• Survey results indicate a growing parental awareness of new assessment practices. 

• Students are becoming more aware of the assessment for learning strategies being 
used in classrooms, and many are asking questions such as, “What’s the target?” 

• Student engagement results on the BarOn Emotional Quotient Inventory show a 
marked growth in grade 5 students, a less significant growth in grade 8 students, and 
no noticeable growth in the grade 1 1 group. 

• Attendance levels of selected grades in Student Engagement have remained basically 
unchanged. 

• Core subject success for the grades 5 and 8 groups exceeded the target, while the 
grade 1 1 group had no noticeable change. 



Figure 10.3. Results of AISI Cycle 3 



The Student Engagement Project originally grew from a High School Completion Project 
idea. Research indicated that students become disengaged at school at an early age, so 
waiting until grade 9 or 10 to intervene might have little or no effect on high school 
completion numbers. The Project Steering Committee approached this issue by including 
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students from grades 4, 7 and 10 in the first year of the project, with the intent to follow this 
same group for the duration of the project, at which time they would be in grades 6, 9 and 12. 
Interventions were put in place at various levels. Some were school-wide, others were based 
in certain classrooms while others dealt with individual student issues and concerns. Data 
from three sources was used to track the progress of the students: Provincial Achievement 
Test marks, C-CAT 41 scores, and BarOn Emotional Quotient Inventory 42 results. In addition, 
other factors such as behavioral issues, attendance, medical concerns, home support, and a 
history of attending many different schools also provided much needed soft evidence in 
identifying and assisting students in their quest to complete school. 



Building Leadership Capacity 

AISI has fostered the growth and development of leaders at all levels. From the first 
cycle until the present, Pembina Hills has undergone an evolution in building leaders. 

During the first two AISI cycles, the demo-coach model of teacher development was 
widely practiced. In conjunction with this, the Lead Teacher model evolved. Three 
division classroom teachers were supported in acquiring expertise in the Balanced 
Literacy Program through Edmonton Public Outreach and the University of Alberta. Of 
these individuals, one is now an Associate Principal, and another has received the 
training as a division Assessment Coach. Training as facilitators of the 6+1 Writing 
Program was offered to a group of seven teachers, and of these, two have continued to 
provide inservices and classroom facilitation of this process within the jurisdictions and 
to divisions beyond Pembina Hills. To date, over 75 teachers have received training in 
providing the Early Reading Intervention (ERI) program to struggling readers in grades 1 
and 2. Of these trained teachers, one is now a trainer for new teachers in Pembina Hills, 
and each school has a designated ERI Lead Teacher. All elementary language arts 
teachers have been given the necessary training and support for implementing Balanced 
Literacy in their classrooms. 

As new teachers take their place in Pembina Hills, mentors are provided, and professional 
development inservices as well as classroom support is provided. The Learning Teams 
model that has been adopted for 2008/2009 in the Classroom Assessment with Purpose 
AISI Project, which is being spearheaded by three recently trained coaches in the Rick 
Stiggins model of Classroom Assessment for Student Learning. This training of learning 
coaches promises to serve the division well beyond the current AISI cycle. The job- 
embedded model of providing teacher professional development is employed in the 
Learning Teams structure as teachers have become leaders who regularly visit classrooms 
to offer support and encouragement. Assessment specialists have been named to serve the 
division needs through the Alberta Assessment Consortium. Although this body is not 
directly linked to the AISI project, it is because of the AISI project that these specialists 
have been installed. 



41 Canadian Cognitive Abilities Test. (1997). Toronto, ON: ITP Nelson Publishers. 

42 BarOn Emotional Quotient Inventory. (2002). Toronto, ON: Multi-Health Systems Inc. 
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Administrative leadership has also been developed in conjunction with AISI projects. At 
the onset of AISI, collaboration was a requirement in setting directions that the projects 
would take, and most schools sent their principal to the planning meetings. Idea sharing 
and a very concerted effort to provide projects which would be most beneficial to student 
learning were the foundations on which decisions were made. 

During AISI Cycle 2, all school-based administrators participated in a two-year school 
improvement program, Planning for School and Student Success. This program was 
incorporated as a leadership component of Cycle 2 projects as feedback indicated that 
when school leaders actively supported the AISI projects, there were better results. With 
that in mind, Wayne Hulley was contracted to work in the division, in an effort to 
improve school leaders’ participation in AISI. Over this time, the research on effective 
schools was used to address the cultural needs of schools as teams of principals, associate 
principals, and designated classroom teachers met to plan and implement strategies to 
meet the specific needs of individual schools. The use of SMART 43 goals was instituted 
as a means to ensure alignment of school purpose, vision, data and learning. SMART 
goals continue to be the means by which schools outline their work in their Education 
Plans, and teachers use them in Professional Growth Plans as well. Further to the work 
with Hulley, administrators have also participated in the Seven Habits of Highly 
Effective People workshop, classroom walk-through training with an assessment purpose 
and focus, and will soon participate in a two-day administrative conference wherein 
Great Leaders, Great Teams, Great Results will serve to deliver further skills in 
leadership. As a further example of the division’s determination to be self-sufficient, a 
team of five Pembina Hills trained administrators will facilitate this workshop. 

AISI projects have contributed to the building of or adding to administrators’ toolboxes 
by supporting an annual Administrator’s Conference, facilitating the development of 
Administrator Professional Portfolios, supporting the engagement of administrator 
growth through professional reading, and building accountability measures into these 
efforts. 

The AISI Assessment Project has served as the catalyst for ongoing administrator 
professional development. During the 2007/2008 school year, under the direction of the 
superintendent, monthly admin meetings devoted a portion of the day to read, reflect, and 
discuss Breakthrough (Fullan, Hill, & Crevola, 2006), a guide to the process of school 
reform. Discussions served to inform and inspire school leaders to more readily assume 
the role of instructional leader, and was a further element contributing to instructional 
leadership and the work initiated by Wayne Hulley. During 2008/2009, the admin group 
has embarked on a study of Ahead of the Curve (Reeves, 2007), a collection of research 
by leaders in the field of formative assessment. 



43 S - specific, M - measurable, A - Attainable, R - results oriented, T - time bound. 
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AISI Sustainability in Pembina Hills 

Early Reading Intervention, Extended Kindergarten Screening, Balanced Literacy, 6+1 
Writing, Comprehension and Communication, and Numeracy have continued in Pembina 
Hills through dedicated funding from the division. Results continue to support the 
necessity of having such programs available for not only the at-risk learner, but also the 
regular student. 

A direct allocation of funds from the division ensures that new teachers are inserviced, 
worked with at-elbow in classrooms, and supported in implementing various literacy and 
numeracy programs. It is an expectation division-wide that schools are using Balanced 
Literacy in all K to grade 6 classrooms, that all grades 1 and 2 students at-risk in their 
literacy development will receive Early Reading Intervention, and that kindergarten 
classes will be funded for administering the Pembina Hills locally developed Kindergarten 
Screening Tool in January each year. All of these programs have continued to support the 
literacy development of elementary aged children, and through them, division results have 
been consistently maintained or have improved. 

The original group of Comprehension and Communication teachers has remained largely 
intact, has gathered new members, and is provided with opportunities to continue their 
development in aligning instruction and assessment as they strive to focus on student 
learning. 



Data Analysis and AISI Projects 

Data analysis is engaged in extensively to inform instructional practices in Pembina Hills 
and supports efforts of improvement. Before AISI, the division had a day designated for 
schools to review and study the PAT and Diploma Exam results of the June writing. 
Historically, this was oftentimes viewed as a “teacher and school evaluation”, but AISI 
has validated it as a necessary component of action research. The student results that are 
a valuable measure in the AISI projects have made teachers reconsider their assignment 
as a craft that can continually be improved, with the extensive professional learning 
support offered by the AISI projects. 

Analyzing data has assumed a level of accountability at not only the individual teacher 
level but also at the division level. Rather than viewing data analysis as a reflection of 
how well or how poorly schools are performing, strategies are implemented with the view 
to improving student learning in areas where the data reflect challenges. School staffs are 
now using their formative assessment knowledge to further their understanding of how 
student improvement can be realized relative to the information provided by Alberta 
Education. As schools become aware of their areas of deficiencies in student learning, 
they are able to collaboratively target additional instruction into content or skill areas 
where these deficiencies exist. Teachers delve into the Programs of Study and determine 
the areas where specific practices in specific grades contribute to student learning. They 
also discuss and review ways to ensure that all students are meeting the standards 
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expected of them. As teachers develop in their awareness of what students should know 
and be able to do, they further explore the manner in which instruction and assessment 
are aligned to ensure that students at each grade level are given the opportunity to 
succeed. 



AISI and the Pembina Hills Education Plan 

A review of the Pembina Hills Education Plan for 2008-201 1 reveals the connectedness of 
the School Board, school administration, teachers, community schools, and communities. 

The plan was developed from a core set of ideas that originated from school council 
representatives in an annual forum facilitated by the Board of Trustees. At this venue, local 
trends, issues, beliefs, and perspectives of individual schools were reviewed. All schools 
were represented by parents, staff members, and, in some cases, students. The information 
from this collaborative consultation process was considered as well as data from previous 
Annual Education Results Reports, needs expressed by school administration, and needs 
identified by division administration. The focus for the Education Plan is summed up by 
the following: 

Student Success + Home and Community Engagement = High School Completion 

This focus will be addressed by four strategic priorities: 

• Increased achievement in mathematics 

• Increased use of formative assessment in instructional practices 

• Increased high school completion for students at risk 

• Increased participation of the home and community in public education 

A further review of this document provides additional support for the catalytic nature of 
the AISI projects. For example, two specific trends are directly related to AISI projects. 
Student achievement and the evolving role and influence of AISI are first and many of 
the associated issues accompanying them directly relate to the classroom practices 
resulting from AISI. 



The Alberta Distance Learning Centre 

The Alberta Distance Learning Centre (ADLC) is governed by the Public Board of 
Education, Pembina Hills School Division. As a result of our AISI projects, the ADLC 
program has grown with the division program. In Cycle 1, ADLC projects focused on 
increasing interaction and building relationships between students, parents, and ADLC 
through the use of technology. During Cycle 2, the videoconferencing project sought to 
extend the work of Cycle 1 projects by providing the opportunity for students to 
experience learning in a virtual classroom. The Cycle 3 AISI project, Distance Education 
Student Assessment, parallels the division assessment project in many ways, albeit in a 
different delivery mode. The following points should be noted relative to the impact that 
AISI has had on ADLC. 
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1. What impact has AISI had on ADLC? 

• A chance to create opportunity to find out what works and what doesn’t 

• A chance for seasoned teachers to reflect on their practice and take a fresh 
approach - new energy 

• A chance to work with other jurisdictions in new and innovative ways 

• Focused the school on student learning 

• Tapped into a latent pool of creativity 

2. What did you learn in terms of meeting the learning needs of students? 

• Increasing student choice in assessment is important 

• Students should have input into the assessment process 

• New models of assessment lead to new modes of delivery 

• We can build on what we already have (rubrics) 

• Access issues are related to assessment 

• The need to educate all stakeholders about new assessment practices is key 

3. What enhancements would you suggest for AISI? 

• More certainty from cycle to cycle 

• Training for new AISI leads 

• More flexible cycle/project length 

• Funds to sustain learning, that is, school boards earmarking funding for 
sustainability at the local level 

4. Where will you go from here ( Cycle 4 and beyond )? 

• Keep up the initiative 

• Standing budget line 

• Projects from six months to five years (real change idea) 

• More coordinated use of PD resources - tie AISI to the new distributed PD model 
developed by ARPDC 44 to maximize PD dollars 



What’s Ahead? 

AISI footprints in Pembina Hills are very evident as the division continues to encounter 
success for students and professional learning for teachers, much of it directly linked to 
AISI. The projects have involved purposeful action with a clarity that provides schools 
the opportunity to clarify their priorities, address their individual and unique needs, to 
collaborate, and to find support and encouragement when needed. 

Pembina Hills Cycle 4 AISI projects will continue to provide embedded support for 
classroom teachers. The solid foundation that is being laid in developing leadership at all 
levels and the accompanying improved student results will be strengthened by extending 



44 Alberta Regional Professional Development Consortium. 
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the work that has been started in Assessment for Learning and Student Engagement. 
Moving forward and going deeper into research and application to practice through the 
Instructional Coaches will help to maintain consistency in practice. Professional 
Learning Communities will continue under courageous leadership and a view to further 
capacity building for ongoing student success. 
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Chapter 1 1 

From the Inside Out: An AISI Journey 

Del Lithe and Gerry Varty 
Wolf Creek School Division 



Since the inception of AISI, our projects have focused on both grassroots initiatives and 
top-down mandates, and featured generous helpings of both failure and success. At times 
frustrating, at other times exhilarating, AISI has led us to consider things we might never 
have thought about, and helped us discover things that may have remained hidden. AISI 
hasn’t always helped us achieve our goals of greater student achievement, but we have 
learned more about what student success looks like, and that might be the right first step 
for our journey. 

While we have experienced some divergent success with AISI as a bottom-up initiative, 
and seen some more focused progress with AISI as a top-down initiative, it appears that 
our greatest achievements come when our AISI projects involve leadership and initiative 
from the middle, when they grow from the inside out. This is a story of how we 
discovered that little pearl of wisdom, and where we intend to go with it. 



Cycle 1 

AISI Cycle 1 featured a wide variety of projects in Wolf Creek. Like many other 
jurisdictions, the response from schools for AISI proposals was mixed between a desire 
to try new things and a grab for more resources. Everybody wanted an AISI project. 

All the projects sounded good. They were grounded in research, but the short time frame 
for development meant that buy-in wasn’t very extensive in schools, and that many of the 
complications had not been uncovered in the discovery phase. As a result, most projects 
neglected to build on a foundation of sustainability; the future would have to take care of 
itself. 

We learned a lot in those first few years. For example, math teachers, when given a 
deeper understanding of the math and more constructivist pedagogy, could make math 
more engaging and attainable for students. We also learned, in following years, that 
when those teachers retired, transferred, or were given new assignments, everything they 
knew went with them. Without continual inputs, the project was not sustainable. We 
learned that early intervention programs, if targeted to students with identified needs, 
could make a significant difference in readiness for grade 1 entrance. We also learned 
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that if those supports were not continued, the gains made were quickly lost. We learned 
that changes have to get past simply structure and form; changes need to go deeper, into 
belief and culture, in order to be truly sustainable. 

Finally, we learned that in the end, change is still hard, and if the driving innovation is 
removed too soon, it’s easier to revert to the ‘old’ ways than it is to forge ahead. While 
we learned a great deal from AISI Cycle 1, we never got a chance to put that learning into 
practice before it was over. At the AISI conference in Calgary in January 2003, we were 
told to wrap up our current projects and pick new ones, projects completely different 
from the first ones. At the time, it was quite disheartening; it seemed that AISI was less 
about what we were learning and more about just doing projects; as one of our leaders put 
it, AISI looks like LYNT, TYNT, and NYNT . . . LAST year’s new thing, THIS year’s 
new thing, and NEXT year’s new thing. 

Given all we had learned about the sustaining effects of culture, it appeared that AISI 
might be doomed to an endless succession of three-year cyclical Groundhog Days; we 
were nonetheless emboldened to try to make AISI work. 



Cycle 2 

With little time to plan for Cycle 2, we took stock of what had worked well: 

• Programs with hands-on leadership did better than others. 

• Programs with supports that extended beyond the classroom were adopted faster. 

• Programs that produced artifacts or indicators of progress that could be shared and 
discussed were more likely to motivate participants. 

• Bottom-up changes rarely challenge culture. 

None of our Cycle 1 projects were truly disruptive, or paradigm-breaking, which led us to 
take a larger role in the next cycle. 

To combat the major forces of resistance, and to bolster the forces that seemed to promote 
change, Cycle 2 projects were encouraged to involve multiple sites. Multi-school projects 
involving common themes were created and initiated, along with a few division-wide 
projects and a jurisdiction mandate that all schools would form and work as collaborative 
teams on these AISI projects. 

For the first time, a large-scale restructuring of Wolf Creek’s Administrators Association 
was undertaken; we adopted a governance model based on a Professional Learning 
Community (PLC) structure. Administrators spent large amounts of time learning about 
PLCs and working at the business of changing the cultures of their schools from isolation 
to collaboration. Multi-school and division-wide projects were also assisted in their PLC 
efforts by district personnel, while the administrators of single-project schools worked at 
helping their staffs learn the new paradigm. 
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Through this all, we spent time conceptualizing a vision and mission for Wolf Creek to 
follow; building upon what we had learned, honoring the work we had already done, but 
audacious enough to give us a shared goal for the future. We collaborated with teacher 
groups, administrators, parents, and students. We studied, read, and watched each of our 
AISI projects to learn which practices were promising and which ones were uncovering 
deeper issues. We built sharing mechanisms, file handling and communication 
structures, videoconferencing and data-sharing technologies, and looked for every 
accelerant we could find. 

We reached out to other jurisdictions in central Alberta, seeking to grow laterally, to 
build on what they had learned, as well. That increase in lateral capacity enabled us to 
agree to work with the Central Alberta PD Consortium (CARC) to host the first annual 
Leading and Learning conference in Red Deer in November 2005. Featuring Michael 
Fullan, Wayne Hulley, and many others, this conference was attended by almost 100 
Wolf Creek teachers, and marked the start of a successful tradition in Zone 4. 

We learned a lot in Cycle 2, and all of it went into our planning for Cycle 3. 

• Again, we learned that leadership is critical to the change process. While we had 
invested a lot of time and effort into getting our principals ready to assist with the 
PLC -based changes in their schools, they are busy people, and the demands of their 
jobs took away from their effectiveness as change agents. Schools with greater 
teacher leadership evident in the projects were much further ahead than schools 
without a teacher-leader component, leading us to believe we needed a new layer of 
leadership in each school, one not compromised by day-to-day demands of the 
principalship. 

• Critical mass is important; simply put, the higher the percentage of people involved in 
the initiative, the more likely it is to be successful at the school level. 

• Common purpose is key to mapping change. Schools involved in district or inter- 
school projects seemed more driven to share and compare their results. Schools in 
isolated projects tended to place less emphasis on data or artifacts, and more on 
personal perceptions. 

In a very real way, Cycle 2 formed the foundation for the success we are currently seeing 
in AISI. Not only did we develop our PLC models in Cycle 2, but we used the time to 
invest in the future. Planning for Cycle 3 was completed long before funding for AISI 
Cycle 3 was announced. By the end of Cycle 2, we had made the critical decision to 
invest in ourselves, and that meant planning to move forward with our plans even though 
we didn’t know if AISI would be renewed. We completed our graphic representation for 
the path our work would take over the next six years, and spent a lot of time taking it 
around to the schools, speaking about it and dissecting it with various leadership and 
input groups, refining the vision of what a high-performing school jurisdiction would 
look like, and planned to move ahead. There was some resistance to the idea of having a 
district-wide project, and many people initially wanted to start with Instructional Design, 
but we knew we had to examine and perhaps break many existing assessment paradigms 
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before we could make meaningful changes to Instructional Design, and fortunately most 
people agreed. This plan had been built from the top down, the bottom up, and the inside 
out. Everybody had a chance to give input and ideas to the framework, and everybody 
knows what it looks like. 



Cycle 3 

When AISI Cycle 3 was announced, we were ready to move ahead. We submitted our 
AISI proposal and got to work. Our first action was to pursue our need to create a 
middle-leadership team - a leadership group composed of teachers, people to take charge 
of the AISI project in their respective schools but still act with a district focus. We held 
information meetings, sent out literature, and explained in depth to anybody who would 
listen what the duties would be for the candidates we sought. The design was simple; we 
would provide release time (an allocation of 0.2 to 0.8 Full Time Equivalent (FTE) to 
lighten their teaching load - calculated on a sliding scale based on the number of teachers 
they had to work with in their site), and each school was also given a collaborative 
allowance based on the cost of two days of substitute time per FTE teacher. As teacher 
leaders applied for these positions, we were not surprised to see that they were very eager 
and well respected, a blend of young and old, from all subject areas. Interviews and 
selections were made, and the AISI Lead Team was formed in the spring of 2006, and 
ready to start in the fall of 2007. 

These leaders subsequently received professional development in change theory, facilitation 
skills, presentation techniques, assessment literacy, and given subsequent opportunity to 
interact with experts like Rick Stiggins, Ann Davies, Damian Cooper, Cassandra Erkens, 
Wayne Hulley, and others. They went to conferences sponsored by the Alberta Assessment 
Consortium (AAC), Reaching and Teaching (CRC), and Leading and Learning (CARC). 

As well, they have been involved in book study groups and several have undertaken their 
master’s degrees as a result of their involvement. These leaders continue to grow in their 
confidence and capability, and as they grow, exert a greater and greater influence on their 
schools. We bring them together every month to work with them and foster a sense of 
community, and encourage them to work with each other in and between schools. Many 
take advantage of the opportunities to travel to other schools and jurisdictions, to see other 
classrooms, and to gather ideas to motivate and grow their own staffs. 

Over the past three years, this team of 36 dynamic ‘middle’ leaders has graduated 8 
members to school administration positions, and continues to attract some of our most 
energetic people, who join to give of their time and knowledge in service to their school 
community. It’s hard work, but it’s a chance to make a real difference, and their 
contributions are making AISI work for us all. 

As Cycle 3 draws to a close, we are pleased that the Central Alberta Regional Consortium 
is getting ready to host its fourth annual Leading and Learning conference in Red Deer; 
for the third year in a row over 350 of our teachers were in attendance (November 20-22, 
2008). Our School Board has supported us in this venture throughout Cycle 3 by 
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declaring the day of the conference a jurisdictional PD day (allowing all staff to attend if 
they choose, with no substitute teacher costs), through financial supports from their own 
budget, and by attending the conference themselves. This is a huge accelerant for us, 
giving all teachers the opportunity to interact with the same experts, to envision the same 
ideas, wrestle with the same concepts, and to develop a common language around the 
changes we are striving to create. 

This year, our project is seeing many schools changing their report cards, grade books, and 
communications mechanisms; this change is not done lightly and is a very encouraging 
sign, as teachers are recognizing their new assessment methodology no longer fits within 
yesterday’s reporting paradigms. Some things come down from the top, others grow up 
from the bottom; we hope that this tension between management control and innovative 
freedom will continue to give our AISI projects life. 



Transitions 

We spent a lot of time, in creating our design framework for AISI, in looking over the 
nature of the transitions between cycles. It seems to us that there are some design flaws 
in the provincial AISI construct, flaws that are inherently detrimental to the intended 
goals of AISI. 

Three-year Cycle. The arbitrary and artificial nature of the three-year cycle dictates that 
change efforts have a finite end point, that they end at a time dictated by a calendar, not 
by culture. This artificiality empowers resistance and systemic inertia, letting people 
think they can ‘wait out’ change. In our model, we have built in time to allow for a 
transition period between cycles, a time when some schools can begin moving ahead and 
others can lag behind to consolidate or complete their efforts. By merely changing the 
lens, we are focused on learning as a constant, and time as a variable, a concept that 
Bloom advocated in the 1960s. 

Project Mandate. The mandate to take on something new, rather than build upon existing 
initiatives and progress gives rise to the notion that AISI is in fact an advanced epidemic 
of ‘projectitis’, and that the purpose of initiatives is to occupy one’s time for three years 
and then be forgotten. Our project framework anticipates and alleviates this by creating a 
nine-year project window enclosing three-year phases of change and focus within it. The 
three phases are seen and understood to be part of an overall vision for success in Wolf 
Creek. When practices are introduced strategically, and in familiar contexts, teachers are 
more comfortable with change. 

The Nature of Data. Sometimes, the data we collect can hide as much as they reveal. 
Collecting data from provincial tests, exams, or surveys only tells us about the things the 
instruments were designed to measure, and without careful scrutiny, can lead to some 
very misleading conclusions. Data gleaned from instruments designed to produce a 
stratified distribution will (if taken at face value) provide reinforcement that such a 
distribution is the only logical conclusion, something our AISI projects are (hopefully) 
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designed to disprove. Positive data for an AISI project should show changes in targeted 
areas, demonstrate a discernible difference from random occurrence, and should be useful 
diagnostics for moving the project forward. Data that cannot meet those identified needs 
are dangerous to act on; at the least, they need to be used with caution. ‘True’ data for a 
change initiative should be triangulated; perceptual data, qualitative data, and quantitative 
data should all tell similar stories. What you measure, see, and feel have to be in 
agreement before substantial changes are made or before evaluations are formulated. 



Conclusion 

AISI in Wolf Creek has become part of the culture - the way we do things. Over the past 
nine years, we have learned a lot about school improvement, change, systems thinking, 
and about ourselves. We still have lots to learn, which speaks to the capacity of our 
people and our systems to grow and adapt; in the words of Robert Frost, “there are miles 
to go before I (we) sleep”. 

The successes we are seeing in our projects are in a very real way, the result of the 
failures we experienced early on. What AISI gives us is a license to fail with purpose in 
mind, an opportunity to learn not only what does work, but what doesn’t. More 
importantly, AISI gives us a mandate to ask why things either work or fail. AISI is not 
only a search for effect, but for cause, and that’s an important distinction for somebody 
who is tired of wandering around the wilderness, being part of seemingly unconnected 
change initiatives. AISI is about finding purpose, about discovering that we make a 
difference, and about proving that what we do matters. Our framework and model for 
systemic improvement is, we think, an innovative way to provide structure and continuity 
to what is necessarily a messy, adaptive process, and gives us a way to see and 
understand where we’ve been, where we are, and where we’re going. 

In Wolf Creek schools, the changes we’re experiencing are more than skin-deep; they run 
through our entire organization, from the bottom up, from the top down, and most 
importantly, from the inside out. 



Del Litke is Deputy Superintendent of Schools and Gerry Varty is Director of Instruction/ AISI Coordinator, 
Wolf Creek School Division. They can be reached at cdlitke@wolfcreek.ab.ca and gvartv @ wolfcreek.ab.ca . 
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Chapter 12 

Looking Back / Planning Forward 

Judy MacKay and Lucy Miller 
Calgary Catholic School District 



This chapter includes four sections: key components, structures and learning from 
previous AISI cycles; direct input from the District Leadership Council in relation to 
potential future directions; recommendations related to observations arising from the 
AISI Colloquium and an overview of how the district’s current context and culture will 
support AISI Cycle 4 project development. 



Previous AISI Cycles 

Since its beginning in 1999, the Alberta Initiative for School Improvement (AISI) has 
moved through a series of project/initiative cycles reflective of its growing maturity. 

Each of the previous three cycles have been characterized by a particular ‘lens’ that has 
captured the nature of the work undertaken at the provincial, district, school, classroom, 
and student levels. 

The move from foundation (Cycle 1) to integration and sustainability (Cycle 2), and most 
recently collaborative inquiry (Cycle 3) has served to frame each cycle for stakeholders. 
Over the course of the three cycles, key concepts woven throughout, and familiar to all 
involved in AISI projects, have included collaboration, leadership, ownership, 
communication, partnership, and most importantly, student achievement and success. 

Perhaps it is time, as we move into the fourth cycle, to move beyond the frame of a 
project/initiative-based model to one of truly embedded practice where there is a breaking 
down of the artificial silo in which AISI has been contained. This might be similar to the 
move at the provincial level with the Student Health Partnership (SHP) where ‘project’ 
and ‘initiative’, which suggested a temporary and isolated status, were eventually 
eliminated. Referring to the work as simply the Student Health Partnership has served to 
support the view of SHP as an embedded practice, a collaborative way of ‘doing 
business’ while maintaining the structures, specificity of funding, annual reporting, and 
accountability that contributed to the success of SHP when it was an initiative. 

Most of the existing AISI components have stood the test of time. They have successfully 
supported innovative thinking and were an embedded part of the presentations and 
conversations at the AISI colloquium. These foundational components have been refined 
over the past nine years, and should remain embedded in the next cycle of AISI. 
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• Partnership among teachers, superintendents, trustees, business officials, universities, 
parents, and government with a focus on developing positive working relationships 
with the purpose of identifying and implementing strategies, and practices that 
provide long-term benefits focused on student success. 

• AISI is a catalyst for change linked to student success. What would it look like from 
the perspective of the student if the best were to happen? While this is a subtle shift 
in the ‘lens’ brought to the collection and analysis of data from what the teacher 
‘does’ to what the student ‘experiences’, it may serve to support increased attention 
on how professional development, pedagogy, and delivery actually make a change. 

• A single-minded focus on student success must include careful consideration of 
differentiated learning, and particular student populations for whom current data 
suggest there are unique challenges and barriers that must be addressed. 

• Flexibility to weave together provincial mandates, criteria, and requirements balanced 
by respect for and encouragement of local decision making, project design, and 
response to specific circumstances. 

• Collaboration and communication are areas with powerful potential to support the 
continuous improvement that is one of the hallmarks of AISI conceptually. 

• Evidence-based practice combined with commitment to data analysis focused on 
continuous improvement and research-based interventions should continue to be key 
components of AISI Cycle 4. 

In addition to these provincially identified key components, a number of basic principles, 

carried forward from the first and second AISI cycles into AISI Cycle 3 in Calgary 

Catholic, also seem to have continued applicability. 

• All schools would be involved in at least one AISI project. 

• Projects would be distributed across the grade levels with a focus on student 
achievement and high school completion. 

• Projects would reflect a top-down and bottom-up balance arising from broad ranging 
consultation and collaboration across the district. 

• Projects would reflect prior learning at either the local or provincial level as shared 
through AISI conferences and online data. 



AISI Colloquium 



79 



Large Scale Improvement 




District Leadership Council Input 



Calgary Catholic - in the context of its current re-imagining process which involves 
effective processes for collecting comprehensive input from a wide range of diverse 
stakeholders, input, and feedback related to AISI Cycles 1 through 3 - sought input from 
the District’s Leadership Council, representing the perspectives of all stakeholders 
including administration, teachers, and parents. Members of this council responded to 
questions of positive impact, challenges, and recommendations for Cycle 4. 



Positive Impacts 

Positive impact for student success arising from the previous three AISI cycles included 

the following. 

• School staffs have increased their tendency to view their planning from a broader, 
holistic manner reflective of the educational needs of the specific learner population 
with whom they work. 

• Specific projects supported clarity and focus geared to creating an action plan tailored 
to local student needs. 

• District-wide capacity has been successfully built through the use of a strategy 
labeled the ‘day six matrix model’ that brings school-based staff, working on the 
same project, together regularly for problem solving, sharing of experience and pro- 
active professional development. Subsequently, individuals involved in the 
collaborative sessions then share with colleagues at their schools. This resulted in the 
creation of ‘on-site champions’ with knowledge and expertise related to specific 
effective learning strategies, professional practice, and effective pedagogy. 

• AISI projects have successfully encouraged teachers to be reflective and intentionally 
thoughtful in relation to critical educational questions while at the same time 
supporting district administrators to consider ways to positively impact change with 
staff as a learning community. 

• The diversity of the AISI projects provided a broad range of opportunities related to 
instruction as well as assessment and evaluation including support for the 
development of specific tools for classroom use. 

• Administrative support, commitment, and leadership are key to ensuring the success 
of specific AISI projects including direct links from district leaders to both school- 
based leaders and school administrators. 

• The deliberate and sustained focus on teaching and learning has resulted in increased 
student achievement. 
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• The opportunity embedded within AISI for shared collaborative goal setting broadens 
professional conversations, increases awareness in relation to effective instructional 
strategies, and draws teachers into the excitement of new ideas with a demonstrated 
track record. 



Challenges 

The District Leadership Council identified the following challenges through small-group 

discussion and completion of feedback sheets that captured key ideas in response to the 

questions provided by Alberta Education. 

• While diversity has positive impact it also presents challenges in terms of stretching 
resources. Given the complex reporting needs, a wide range of projects can be 
challenging in terms of management and accountability. 

• The three-year cycle has seemed short in terms of making key changes. It has made it 
difficult for schools to adapt since schools require opportunity and time to adapt to the 
criteria and focus of each new cycle. 

• The AISI project has been very beneficial to the school; however, many school initiatives 
are targeted to higher needs of the school/district and extend over and beyond three years. 
More time is needed. 

• Sustainability of projects that prove to have a positive impact is difficult since it is not 
always possible to reallocate basic funds to support new initiatives since they are needed 
to maintain basic operations. According to one participant, “Some projects target a 
particular area in the school/district (CCC, D2L, even special support) that becomes 
integral to school function yearly and so funding becomes difficult year to year.” 

• The time required to collect data and report on them continues to be complex and 
mammoth in a large metro board. “It took a long time for us to understand what we 
needed to do - in order to fulfill the AISI needs.” Another said, “Sometimes it seemed 
that AISI staff spent a lot of time doing paperwork and collecting data rather than 
working with children or staff. Reporting and data gathering requirements are 
expensive - these should be relaxed. 

• There is a need to simplify and streamline reporting. Current reporting requirements 
are not really meaningful at the classroom level. 

• Some staff resistance has created barriers to implementation in some locations. A 
related challenge has been finding the time for AISI leaders to make presentations 
and lead professional development to staff given the many topics and PD needs that 
schools are balancing. 



AISI Colloquium 



81 



Large Scale Improvement 




• Communication both in the district and across the province has been difficult due to 
the sheer quantity of information although AISI conferences, the Clearinghouse, and 
publications have been helpful. 

• While in some cases three years have been too short, there is some need for flexibility for 
specific schools to be responsive to shifting student needs as well as changes in staff and 
administration. 

• Coordination of large projects can be challenging in terms of time limits, points of 
contact, and integration into the work of the entire school/district. Sometimes the day- 
to-day realities of school make it difficult to maintain the focus on the AISI project. 

• The current reporting process has not always supported an emphasis on the qualitative 
aspect including the value and positive impact of face-to-face interaction and anecdotal 
reports. 

• Staff training, particularly for AISI leaders, can be a challenge when there may be 
annual staff shifts. 

• The short term nature of funding, even over three years, made it difficult to sustain the 
projects as they were originally conceived. 

• It was difficult to find the necessary time for effective and sustained staff collaboration. 

• Adapting to increasing student numbers in need of support stretched the resources over 
the three-year period. 

• Transient student populations in certain schools made it challenging to collect data and 
to gauge the success of a particular project. 



Recommendations 

The District Leadership Council also made recommendations for AISI Cycle 4. 

• Continue the commitment/opportunity for schools to design AISI Cycle 4 around the 
concept of encouraging schools to address the challenge of ownership by increased 
support for local schools to research and direct their projects. This would increase 
accountability and ownership with the projects. 

• Continue to gather and share best practices at the provincial level. 

• Continue a “grassroots” capacity building model in order to continue staff development. 
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• Recognize the diversity of needs between districts in the province as well as between 
schools within a district. 

• The Calgary Catholic re-imagining process provides a specific structure that allows for 
broad- based input from many stakeholders leading to specific action plans reflective of 
their input. 

We believe these observations and recommendations, developed in the context of Calgary 
Catholic’s AISI projects, are common across the province. We now focus on capturing 
what we perceive to be commonalities as well as a focus on recommendations for the future. 



Observations, Recommendations, and Future Directions 

The AISI Colloquium provided opportunity for excellent conversations with a broad 
range of partners and stakeholders. Exploration of previous cycles and current realities 
related to AISI reflected many common themes, perspectives, and understandings shared 
across the province even though the realities of the various districts were widely 
disparate. 

Key themes, repeatedly reflected by participants, confirmed the impact of AISI as an 
effective mechanism and structure which has positively impacted the educational culture 
at the school, district, and provincial levels. Recurring themes closely reflected Calgary 
Catholic’s AISI experience which has included the following. 

• Supported and documented improvement in student achievement 

• Active creation of opportunities for professional capacity building and improved 
collaborative planning 

• Encouragement of more focused interventions that address barriers to student learning 

• Delivery of quality PD through both traditional and innovative structures 

• Support for the development of professional learning communities 

• Increased reflective practice, data-based decision making, professionalism, shared 
responsibility, and leadership 

School jurisdictions and AISI partners reported similar benefits; they also identified 
common challenges to be carefully considered as planning for AISI Cycle 4 proceeds. 

• Ongoing challenge of meeting the needs of an increasingly complex and diverse 
student population 

• Sustainability of the innovative ways of ‘doing business in education’ that are 
identified in the context of AISI but that are negatively impacted by the need to 
grapple with the limits of funding and resources 

• Consistent reports, particularly from the school level, about the amount of time, 
energy, and effort required for accountability and management that were often 
perceived to undermine the time available to work directly with students and engage 
in teaching and learning 
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• Meeting the challenge of day-to-day time constraints, staff resistance, and the lack of 
flexibility created by the three-year project cycle 

Against this backdrop of commonalities and challenges, the following observations and 
recommendations may be helpful in guiding the development of the framework, 
processes, and specifics of AISI Cycle 4. 

In the past three cycles, AISI has been identified as a bold approach to supporting the 
improvement of student learning by encouraging teachers, parents, and the community to 
work collaboratively to introduce innovative and creative initiatives based upon local 
needs and circumstances 45 . This initial vision as a foundational statement continues to 
provide the compass by which we may collectively steer our course in planning AISI 
Cycle 4. However, ‘morphing’ from the original concept of AISI Cycle 1 is normal, is to 
be expected, and should be celebrated. Shifts in how AISI looks, in its parameters, 
organizational structures, and accountability structures should be expected and celebrated 
as evidence of progress, of learning, and of innovation. It should be viewed as a good 
thing that reflects the wide-ranging ‘learnings’ that have occurred across the province 
since 1999. 

An implication here is that while it is necessary to recognize and value the foundation 
and groundwork provided by the previous three AISI cycles, it is equally important to 
avoid being ‘tied’ to previous visions and directions. The whole point of AISI is to 
move forward. As a result we need to be cautious about how closely AISI Cycle 4 is tied 
to the original parameters of the original vision. It is a natural process for a vision to 
evolve and shift in response to what is learned. 

During the first phase everything was new. There were wonderful opportunities presented 
for exploration, for breaking new ground, and for energizing educators to look at their 
practice with an eye to the benefit of innovative and improved professional practice upon 
student achievement and success. Now however, we are at the end of Cycle 3. An immense 
expertise has been built in relation to professional practice and there is a broad base of 
understanding about what is working and what needs to be done to continue to evolve. 

In the world of education and research, nine years of successive three-year studies is not 
that long when one considers the length of time required to significantly impact change at 
the student, classroom, and school levels. At the same time, it is important to bear in 
mind that education is a process of continuous improvement. This makes it important to 
celebrate progress and success while at the same time keeping in mind that it will never 
be perfect. “We will never be at the end of the journey to support student success and 
achievement.” What AISI facilitates is a journey not a final destination. 

Critical to its continued success is AISI’s ability to be responsive to student learning needs. 
This focus on student learning is what needs to be continued. Everything needs to be 
linked to student learning and there needs to be publicized, high profile understanding 



45 From the AISI website http://education.alberta.ca/admin/aisi.aspx . 
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and explanation for stakeholders that student success involves more than academic 
achievement. Although AISI can’t account for student achievement entirely, it also can’t 
operate separate from it. At the same time that we explore how to reduce and manage 
data collection more effectively within AISI projects, it is equally important not to lose 
sight of the focus on improved student achievement. It is important to develop AISI 
projects that are not ‘just add-ons’ but are carefully and fully integrated into day-to-day 
practice and are part of ‘doing business’ in terms of quality program delivery. 

In addition, direct links between AISI ‘learning’, student achievement, planning, and 
decision making need to occur not only at the school and district levels but also at the 
provincial level. In Cycle 4, it will be important to be intentional and deliberate about the 
connections between the school district and provincial reality, between distinct areas such 
as teacher induction, high school completion, the Setting Direction for Special Education 
initiative, and many others. For Calgary Catholic this means consciously moving away 
somewhat from the project-based approach to opportunities for a fluid response to the 
data that will be collected over the next three to five years in the context of Cycle 4. A 
key learning from the previous cycles, that needs to be operationalized in Cycle 4, must 
be that increased responsiveness and ongoing flexibility to meet the shifting demands of 
the classroom and schools need to be carefully and directly embedded in the structures 
and requirements that define the next cycle. 

As schools, districts, and the province, we have to make systemic thinking a normal way 
of doing things. This means putting processes and strategies in place that facilitate 
opportunities for stakeholders to come together. When we do have those points of 
contact, we have to ask ourselves how our day-to-day lives will be different because of a 
particular initiative. Systemic change is change that is deep-rooted, that affects how we 
operate. 

In this context, it is important to ensure that AISI Cycle 4 reflects an understanding that 
various stakeholders and partners have shared but also different responsibilities. A key 
question related to this concept is, “How can all partners be encouraged to take 
responsibility for building the capacity and improving the quality of the performance of 
their own stakeholders, their own group?” That is, trustees and boards examine their own 
practices, district senior leadership examine theirs, school-based administrators theirs, 
classroom teachers theirs, and parents theirs. In examining previous AISI cycles, and 
many other educational initiatives, there is always a tendency for stakeholders to look at 
other stakeholders, to identify what another group could do better to meet the needs of 
children. Consideration of what and how each stakeholder group and partner - trustee, 
superintendent, principal, and teacher - might focus on to best improve the part of student 
success for which they are directly responsible would translate perhaps into encouraging 
groups to look within their own group, within a project where appropriate. 

Throughout the many conversations at the colloquium, many barriers and challenges 
were identified. In grappling with barriers and challenges, it is important to ensure that 
collectively and individually trustees, superintendents, district and provincial staff move 
towards a focus on what we can control rather than being discouraged by what we can’t 
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control. Each partner and group needs to take responsibility for its own actions. They 
need to avoid looking to others to save us and to refrain from blaming others. We all 
have a responsibility for student achievement and the responsibility of each of us is 
different. Other key questions for groups are: “How can we be better superintendents, 
parents, teachers, trustees, principals? How can we, operating appropriately within the 
scope of our different roles, support increased student achievement directly?” 

As we consider how to move forward into Cycle 4, a key question is, “How do we protect 
innovativeness and risk taking while still acting responsibly?” At the same time, it is 
important to bear in mind that innovative does not necessarily mean a particular school 
jurisdiction, school, or classroom teacher has not used a particular innovation, structure 
or strategy previously. Rather, innovation might refer to the culture being built. It could 
refer to how a particular district, school, or teacher responds to a way that is successful 
already. It could be how they react to a changing reality to explore how existing 
structures, strategies, and initiatives work in a ‘new reality’. 

On a practical note, the final recurring theme for which a specific strategy is required is 
how funding beyond AISI can be designated for sustainability of selected initiatives over 
time. In some cases, the concept of identifying, through AISI, alternative strategies to 
deliver effective education that can replace other models continues to be valid. However, 
it is equally important to recognize that enhancements to the learning environment, made 
possible through AISI, may also just involve additional resources that require sustainable 
funding in addition to what already exists. These two components of implementation of 
innovative approaches need to be recognized and addressed. 



Conclusion 

Planning, development, and delivery of any initiative, large or small, within a school 
jurisdiction is directly impacted by the existing culture of consultation, leadership, and 
operational management found in the particular organization. For Calgary Catholic in 
2008/2009, this means relying upon the ‘lens’ brought to consultation and collaboration 
by the district’s commitment to the framework of Shepherd Leadership at all levels of the 
organization. 

Since beginning the process of defining Shepherd Leadership in the spring of 2007, Calgary 
Catholic has actively re-defined and re-shaped the principles, processes, procedures, 
strategies, and structures used to support shared understanding, common language, district- 
wide alignment and ownership, and focused collaboration related to student success and 
achievement. This current reality opens the door to rich conversations in the coming 
months as the topic of developing suitable AISI Cycle 4 projects for implementation in 
September 2009 is brought forward throughout the district for consideration and input. 
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Cycle 4 presents opportunity, it presents challenge, it presents an invitation to continually 
re-imagine - What does it look like when the best happens for the children in our care? 

It requires that together as members of the educational community we move from ‘me to 
we’. In answering the question of what next in AISI Cycle 4, it can’t be all about ‘me’ 
as a superintendent, a parent, at trustee, a teacher, a principal. It must be about the 
student - the learner for whose success we are all charged and responsible. This requires 
trust, collaboration, and a genuine commitment to shared responsibility. 



Judy MacKay is Superintendent, Specialized School Programs/Instruction/Religious Education, and Lucy Miller 
is Chief Superintendent, Calgary Catholic School District. They can be reached at Judv.MacKav@cssd.ab.ca and 
Lucv.Miller@cssd.ab.ca . 
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Chapter 13 

Systemic Change for School Improvement 

Brian Andjelic 

Prairie Rose School Division 



AISI Impact on Division 

AISI has had a huge positive impact on Prairie Rose School Division. Three key areas 
are research-related learning, fostering a positive culture, and encouraging a culture of 
risk taking. 

Research-related Learning. At the district level, school level, and even the classroom 
level, we are much more likely now to consider a research base for the work we do. 

There still is some challenge to refine the research as we consider the true meaning of 
“quality” research. We now belong to a number of education data bases. We have 
refined the professional journals we subscribe to, and have involved a number of staff in 
book clubs. Our Professional Learning Council (established late spring between Cycle 2 
and Cycle 3) guides the adult learning strategies and priorities of the division. We 
believe this council will be sustainable for the foreseeable future. Video conferencing 
has allowed us to have “after school” conversations in a division the size of Switzerland. 
Learning communities are alive and well. 

Positive Culture. The learning community (learning team) approach over time has been a 
positive contributor to outgrowing the many pains of four school divisions coming 
together as one in 1995. Some of this, we believe, can be attributed to our AISI model 
that supports division direction and strategies while actively supporting grassroots, 
school-based strategies as well. We have had a very sustainable and loyal AISI Division 
Steering Committee based on the philosophy of how the provincial AISI Education 
Partners Steering Committee is structured. Although we still have much to learn, AISI 
has reduced isolation and increased collaborative sharing. The division provides some 
direction and allows lots of flexibility within our AISI frameworks. 

Culture of Risk Taking. Teachers within Prairie Rose have become more comfortable 
taking calculated risks in their AISI projects that otherwise would not likely happen. A 
few examples include cross-curricular approaches to teaching and learning, multi-graded 
project-based learning, new special needs models, and character and citizenship 
education. AISI funds have done much to foster this attitude among our staff. 
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Lately our people forget whether the conversation is about curriculum, professional 
learning, or AISI. We are most pleased that the dots are being connected. AISI is 
becoming a piece of the integrated problem-solving approaches that occur within Prairie 
Rose. School improvement is no longer really considered outside of the day-to-day 
operations within our division. 



Changes Over AISI Cycles 

We learned a lot about school improvement over the three cycles of AISI. 

Cycle 1. In Cycle 1 there was little focus. The analogy we use is that someone fired the 
shotgun and a huge number (21) of small, very specific projects fell from the sky. Most 
of these were mortally wounded. There was very little sustainability or transferability. 
Our own analysis of Cycle 1 projects is that very few of these projects actually “stuck” 
with the division. 

Cycle 2. As we planned Cycle 2, we realized that a three-year funding cycle was not 
particularly supported by the research. Systemic and sustainable change, especially at a 
division level, simply takes longer. Therefore, we more or less planned for two cycles at 
this time. Much of Cycle 2 was about establishing a positive and collaborative culture 
through learning teams. We felt that we needed to have a kind of a framework for 
establishing priorities and certain processes would foster the culture we were seeking. 
Even though eight themes were selected through a focus group planning approach, lots of 
flexibility allowed for grassroots planning, initiative, and excitement. These themes 
came through an analysis of the gaps in our PRSD Education Plan and through feedback 
from the three geographic focus groups we hosted. Much of the work of the DuFours 
was central to our Cycle 2 project. 

The prologue of our Cycle 2 project plan stated: 

The PRSD project is expected to provide for flexible, effective and lasting improvement 
of staff, student and parent learning and achievement. It is intended to provide for 
professional development that otherwise might not occur. It is also intended to foster a 
spirit of collaboration among all PRSD partners as they learn together. . . . We hope to 
foster positive school climates as our schools take the first steps towards becoming true 
learning communities. 

Cycle 3. During the early days of Cycle 3, the work of Fullan and Collins continued to 
have influence. We felt that we were well on the pathway to continuous improvement 
but that we needed to be more precise. Fairly early in the winter of the final year of 
Cycle 2 we felt that we should take the chance that AISI would continue and began our 
planning. We built on the data from our focus groups in Cycle 2 and put together a 
“think tank” of the entire education team at central office including technology, 
instruction, and special needs. From this was bom our Professional Learning Council 
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made up of representatives from each school, the Alberta Teachers’ Association, 
administration, and the members of the original think tank. The new framework and the 
new battle cry “From Good to Great” were bom. 

Collins (2001) suggests to first “face the brutal facts.” A key strategy was to no longer be 
satisfied with comparing achievement scores with the provincial average. We began to 
track achievement, as measured by Provincial Achievement Tests and Diploma 
Examinations, against ourselves. Once we did this we learned that there were some real 
holes in our achievement. At least some of the strategies have been successful, as we’ve 
made some gains in a number of achievement areas. 



Meeting the Learning Needs of Student 

Hard measures continue to be the greatest challenge, in my opinion, of continuous school 
improvement. The similarities and differences between the effective schools movement 
and the school improvement model contribute to this confusion. We know that what 
“sticks” in one site may not be appropriate for another site. Yet, some high yield 
strategies, such as reading, special needs, and assessments for learning, for example, are 
discussed often in the school improvement literature. 

A fundamental question might be, “What is it we wish to measure - achievement or 
learning?” These terms do not mean the same thing and yet it seems they have become 
synonymous. In social sciences such as education, health, police work, etc. it is very 
difficult, if not impossible, to isolate the many variables at play. It seems to me the 
action research model is a little overrated in our Alberta context. If we wish to travel 
down this path in a more precise manner we will need more support from the experts. 
Most of us in the field are educators, not educational researchers. 

If it’s about raising the bar and closing the gap of achievement, then the strategy of 
largest and quickest improvement is to clearly identify students “at risk” of failure - such 
as not graduating, behavioral issues, social issues, mental health issues, etc. - and target 
strategies that will help these students. 

If it’s about learning, the strategies for improvement become much more complex. One 
strategy in which we have seen more direct evidence of promise for measuring learning is 
a cross-curricular approach taken at Eagle Butte High School. This high school has 
seven feeder schools. Transition is obviously a challenge. The school was unhappy with 
student failure rates in grade 10. The staff felt that this increased the risk of not 
graduating. A key strategy was to identify the grade 10 students coming to this school 
who might be “at risk.” Instead of putting these students into an Integrated Occupational 
Program including English 10-3, the students were offered English 10-2 and were 
immersed with students who were identified as positive role models. This cohort of 
students took English 10-2 first block and Information Processing last block. A major 
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difference was that the key teachers in the project used a team-teaching approach and the 
students’ work flowed between their English 10-2 work and the presentation of their 
work (Information Processing). 

At the same time, however, a significant change to a more meaningful integration 
approach occurred at the site through our special needs department. The good news is 
that the failure rates dramatically decreased, from 24% of grade 10 students failing one or 
more classes in 2006/2007 to 9% in 2007/2008. The bad news is that it is impossible to 
deter mi ne which variables have how much influence over the outcome. 

Between our special needs Pyramids of Intervention, another learning team approach, and 
the lessons learned in AISI, student achievement and student, parent, and teacher 
satisfaction are generally on the rise as indicated in the PRSD accountability report. It’s a 
bit of a leap to conclude this means an improvement in learning. The bottom line is that 
we need more guidance in measuring learning. It seems that we have more data than we 
can effectively use. We need to “weed the data garden.” 



Enhancements for AISI 

“Initiative” invokes fear and eye rolling and “this too shall pass.” The term itself is pretty 
much the opposite of sustainability and connectedness. “Projectitis” is problematic: 

Mental Health Capacity Building Project, Healthy School Community project, project, 
project, project. We barely survived “projectitis” in the first cycle. 

We’re not sure if the debate should be about broad versus deep learning, but rather, what 
is an appropriate balance between the two? The new mission of schools aims high: 
education that is truly for all. “We need to put our energy in the right combination of 
places” (Fullan, Hill, & Crevola, 2006, p. 12). I suggest changing the name to ASIP, the 
Alberta School Improvement Plan. 

Next, let’s have a plan. This next suggestion may not sit well with many of my peers. In my 
opinion, there has actually been too much choice for how to get involved. I understand why 
this has happened and I fault no one for this, especially considering the context of getting 
started. The funding has been appropriate. The risk taking has been appreciated. However, 
after nine years of experience and considering the research that is available, there must be 
some high yield strategies that are proven in pretty much any context. Having a framework 
of choice might not be the worst thing as long as we don’t go too far in restricting how one 
might get involved. I’m not sure keeping school jurisdictions accountable for good 
strategies through “quality research” has delivered on the intentions. What is “quality 
research” anyway? Most of us are only research “wannabees.” 

There is much good work in our current provincial AISI application template. Much of 
the application process is about the how to strategies. This is excellent. What I think is 
missing is the attention to detail regarding the change process. The research that I have 
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read (hope it’s “quality”) clearly does not support a three-year time frame. I understand 
budget and political realities; however, policy should seriously consider the time frame. 
Change that is to stick and be transferred takes a plan that is more futuristic than three 
years. 

We have learned much about sustainability. We should now pay closer attention to 
transferability. Small, individual projects, in our experience, are much more difficult to 
transfer among schools or throughout the division. Our district direction has been much 
more successful in this regard. Our schools have had a choice for how to get involved. 
There seems to be enough variety to engage those who are ready to be engaged. 



Lessons Learned - Cycle 4 and Beyond 

Based on our experience over the past three cycles, we have learned the following: 

• Adopt proven yet innovative practices. 

• Leadership, critical thinking, and problem solving for students and adults. 

• Ready to learn means timely, as needed, and convenient learning as well as paying 
strict attention to the culture of collaboration and continuous improvement. 

• What are research-based, high yield strategies in key areas: literacy, math, wellness, 
assessment? 

• There seems to be a strong relationship between Critical Learning Paths (Fullan et al., 
2006) and Special Education’s Response to Intervention (Ontario Ministry of Education, 
2008). We have developed fairly detailed Pyramids of Intervention for academics, 
behavior, and social skills. However, we think these can be more precise by studying 
these promising practices and becoming more specific with an assessment plan. We 
think these ideas have huge potential across the wide spectrum of student abilities. Not 
everyone can be average, nor should they! 

• The triple P core components - personalization, precision, and professional learning - 
of a breakthrough system (Fullan et al., 2006) appear appropriate to our work. 

• The Fourth Way of Change (Hargreaves & Shirley, 2008) is a framework for school 
improvement that seems to fit with our experience. 

• How can we keep what is working, needs more time to become precise, and inject 
creativity and innovation at the same time? Our division would be happy to discuss 
this further if there is interest in a pilot project to test some ideas in this area. 



Conclusion 

AISI has provided a tremendous opportunity to improve student learning in Alberta. The 
AISI program has married the concepts of effective schools, continuous improvement, 
curriculum, professional learning, and more. It has encouraged a deep and thoughtful 
conversation about education. What is the purpose of schooling? Is it to raise the bar 
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and/or close the gap in relation to academic achievement, wellness, other? What is the 
role of AISI considering this big, broad picture? AISI cannot be an island unto itself. 
Where does it fit in relation to other Alberta programs and policies? 

AISI also provides an opportunity to try new things, to take risks. How can we be precise 
without being prescriptive? How can we be loose and tight at the same time? We have 
to discover new and valid ways of measuring school improvement and especially student 
learning. Never underestimate the role of vision and how that vision is created. Never 
underestimate the good work that results from a group of caring, skilled, and motivated 
teachers sitting around a table with a purpose. 

LEARNING = ENERGY = MOTIVATION = ENERGY 
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Part 3 



External Perspectives 



Major Teaching Strategies in AISI Projects 

0 5 10 15 20 25 30 

Hands-on experiences 
Enrichment 
Differentiated Instruction 
Teaching to Accommodate Learning.. 

Alternative Delivery 
Individual Programming 
Problem Solving 
One-on-one/Small Groups 
Teaming/Cooperative Learning 

■ Cycle 1 ■ Cycle 2 



DREAM 





AISI Strategies I Have A Dream 

Robert Crocker Pasi Sahlberg 



Complex Networks 

Productive Stress 
Sufficient Resources 
Constraints that Enable 





k-lil 


... but about 

connect and 


A decentralized 


f \ \ • 


collaborate ... 


network is NOT 


L\\ L N. 




about command 


* \\A | j * 


... and about 


and control ... 




being 






collectively 




/ * 1 \ / 
V — • 

DECENTRALIZED) 


smarter. 



Complexity Thinking 
Brent Davis & Dennis Sumara 




The Fourth Way 

Andy Hargreaves 







Chapter 14 



Evidence for Educational Policy and Practice: 
What is Credible? What is Useful? 

Robert Crocker 

Atlantic Evaluation and Research Consultants 



What constitutes credible evidence for use in educational policy and practice? Although 
“evidence-based decision making” has become something of a catch phrase in education 
(and in other public policy areas such as health care) over the past decade, and everyone 
seems to want evidence to support their decisions, rarely is the question of what constitutes 
reasonable evidence asked. 

It is important to note what Kuhn (1970) calls “normal science” - the tedious and plodding 
process of gathering evidence to support theory, policy, and practice. Some may be calling 
for a Kuhnian “paradigm shift” in education, in the form of a complete rethinking of what 
constitutes schooling, teaching, and learning. In other circumstances, I might support such 
a call. However, that is not the purpose of this chapter. The reality is that, even under a 
paradigm shift, someone has to continue with the painstaking task of determining whether 
the new paradigm works better than the old one. Even in physics, where the revolutionary 
works of Einstein, Schrodinger, and a host of others in the early part of the 20 th century put 
to rest the prevailing positivist view that the world is deterministic, it remained necessary 
for empiricists to determine if the new theories of relativity and quantum mechanics 
yielded better predictions than traditional Newtonian mechanics. That process continues to 
this day, as evidenced in the continued search for elementary particles and cosmological 
events that will verify the theoretical framework, even as theorists continue Einstein’s life 
work by searching for a new unifying paradigm that will bring together the two main 
physical theories. 

Without going further into the nature of science or the extent to which education should 
be treated as a science, it is assumed that there is increased demand for major policy and 
practice decisions in education to be informed by scientific evidence. There is a clear 
desire on the part of policy makers to move away from intuitive “seat of the pants” 
solutions to complex problems and from policy following the path of least political 
resistance or being driven by advocacy groups. That is not to say that these influences 
are not still alive and well, or that they should be abandoned entirely. However, with the 
increasing sophistication of an educated public, the greater exposure to research on the 
part of educational leaders, and the success of evidence-based approaches in other 
professional fields, the time now seems riper than ever to achieve a greater balance 
between the competing forces that drive policy and practice. 
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Educational Decision Making 

Policy and Practice 

Although this may be obvious, it is useful for the rest of this discussion to make a 
distinction between policy and practice. What I refer to as policy is decision making at a 
relatively high level. Policy is intended to influence a whole system or large units within 
the system. The level of policy making of most interest here is that which would occur at 
the provincial or national level, although other entities such as school boards or schools 
may have policies on various matters within their realm of decision making. 

Professional practice, on the other hand, involves the “on the ground” work of principals, 
teachers, and other professionals in the system. A good part of professional practice 
involves implementation of policy (sometimes faithfully, sometimes not, sometimes 
subversively). However, in a profession, much more so than in a trade or technical 
occupation, there is also usually a strong element of individual knowledge and judgment. 
This is typically derived from the practitioner’s professional training and specialization 
or, in the case of teaching, from disciplinary knowledge in school subjects, pedagogy or 
other fields. However, it is also driven to a large degree by intuition, local knowledge, 
and personal predilections and preferences. 

The fundamental goal of both policy and practice is to improve our ability to achieve 
valued educational outcomes. In other words, I take the pragmatic stance that the focus 
of evidence in an applied field such as education is mainly to allow us to find out what 
works. Although this seems simple enough, this points us to a key issue which may help 
explain why education has lagged other professions in the adoption of an evidence-based 
approach to policy making. The point is that before we can consider how to improve 
valued outcomes we have to know what these outcomes are. In most professions, these 
are fairly obvious. For example, the goal of medicine is to prevent and cure disease, and 
at its core to increase human life span. In medical studies the outcome is almost always 
decreased morbidity and/or mortality. In economics, the outcome is almost always 
financial; make more money, increase productivity, reduce debt, increase stability (not 
that any of these are being very well achieved these days). In sports, the goals are, in the 
words of the Olympic motto, higher, stronger, faster (not to mention more money). 

Education is one of the few professional areas in which controversy continues to exist 
over goals and outcomes. Is our primary goal to increase student learning in certain well- 
defined areas? If so what are these areas? If not, what else should we be trying to 
accomplish? At this point, the argument that the primary goal is enhanced student 
achievement, narrowly defined, seems to be winning at the highest policy levels, as 
evidenced by the proliferation of large scale efforts to measure student performance in 
reading, writing, mathematics, and science. However, there are many who would argue 
that this is much too narrow a conception of the goals of education. Indeed, the mission 
statements of most departments of education in Canada take a much broader view. For 
example, Alberta Education gives the following as its vision and mission: 
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Vision 

The best Kindergarten to Grade 12 education system in the world. 

Mission 

The Ministry of Education . . . inspires, motivates and provides the necessary tools and 
opportunities for every child to attain the knowledge, skills and attributes required for 
lifelong learning, self-sufficiency, work and citizenship. 

Source: Alberta Education (2008). 



I won’t comment on the vision other than to note that Alberta is not alone in making such 
an ambitions statement and to say that later on I will present some evidence on whether 
Alberta is anywhere near to achieving this vision. On the mission, may I take for granted 
that everyone in this room carries this statement around in your pocket and refers to it 
every day as the basis for everything you do in your work? May I also assume that 
everyone knows clearly what knowledge, skills, and attributes are required for lifelong 
learning, self-sufficiency, work, and citizenship? Rhetorical questions both, I expect. We 
could spend the day debating what attributes are most valued in education. However, that 
is not what I have been asked to do. I can only take the position that we do have a good 
idea of at least some core goals, and that, in moving towards the use of evidence-based 
decision making, at least some of these goals are articulated well enough to be measured. 



Factors Influencing Policy and Practice Decisions 

It should be fairly obvious that policy and practice in education are influenced by a large 
number of factors. Some of the most important are: 

• Intuition/belief/common sense 

• Personal views of decision makers 

• Persuasion/advocacy 

• Cost 

• Political ideology 

• Expert opinion/reliance on authority 

• Empirical evidence 

• Theory 

A case can be made that a great deal of decision making in education, particularly at the 
highest political levels, is based on the first four of these. While all of the education 
ministers that I have known have all been fine, thoughtful people, ministers, and 
governments generally, do not usually come to their jobs with special expertise in 
education. Indeed, at the highest levels, it is doubtful if educational policy should be 
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made by the experts - but that is another issue to remain untouched. However, most have 
had, at best, an intuitive vision of what education is about and how the system should be 
supported and organized. At some levels, such as curriculum content or teaching 
methodologies, most ministers are content to leave things to their professional advisors 
(thus reliance on authority). However, on broad structural and ideological issues, such as 
school choice, school board structures, or funding priorities, the seat of the pants (or, as 
Hillary Clinton would say, the pantsuit) still rules. 

We are all well aware of the work of advocacy groups of various stripes and of their 
ability to influence the policy agenda. Indeed, next to intuition, this may be the strongest 
influence on political decisions. Among these groups, none is more powerful than the 
political opposition. Ben Levin, who seems to have been deputy minister of education 
almost everywhere, once observed, “imagine how your work would change if there were 
people whose full time job was to make you look bad. Imagine also that they can be less 
than scrupulous in doing so and that there was a tendency for people to believe their 
criticisms ahead of your explanations” (Levin, 2002, p. 22). This is something that no 
leaders other than those in the political realm have to face. Beyond this, political leaders 
are constantly bombarded with demands from narrow interest groups. Given electoral 
imperatives, it is not surprising that the natural instinct is to try to satisfy these groups, 
while hoping that nobody else is interested enough in the particular issues of these groups 
to notice come election time. 

Cost is always a consideration in policy making. As a case in point, we frequently hear of 
educational expenditures referred to as “investments.” This is based on a well established 
link between education and individual incomes. However, it does not follow from this 
that governments will be able to find the money at any given moment to implement 
programs or that the investment case for all educational innovations is equally strong. 
Again, judgments have to be made and evidence is needed to support these judgments. 

Let us use a specific example to illustrate these points. Class size has been a matter of 
concern and controversy, probably as long as there have been schools. The factors 
influencing decision making on class size can easily be illustrated by the following points 

• It is easy to believe intuitively that smaller class size is “better.” 

• Teachers and parents tend to advocate smaller class sizes. 

• Achieving smaller class size is expensive. 

• Depending on political ideology, governments may or may not believe it is worth the 
cost. 

• What evidence do we have that smaller classes are better? 

• What do we mean by better anyway? What outcomes would we measure to 
determine what is better? 

Given the intuitive sense and the clear direction of advocacy, and the contrary arguments 
around cost, it is not surprising that policy makers would, at some point, ask the obvious 
question, “What evidence is there that smaller classes are better?” This is accompanied 
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by a host of more specific questions such as: “Better for what? Better for whom? How 
small should we go?” As we shall see, although class size has been studied quite 
extensively, a clear picture of the value of moving to smaller classes has yet to emerge. 



Evidence-based Policy and Practice 

The goal of the movement towards evidence-based decision making is to shift the balance 
from intuition/authority/advocacy towards use of evidence. It is important to stress the 
idea of balance because I cannot imagine a situation in any of the human sciences, 
including medicine, under which the views of the persons affected by a treatment or 
innovation would not be considered. The same holds for cost. However, I am arguing 
that a sizeable shift is possible and, indeed, is required, in education, that evidence can 
and should play a much larger role than it has done historically and, taking the argument 
further, that this is the only way in which we can move from what I view as a plateau in 
educational quality and create an education system that is significantly more effective and 
efficient. 



The Nature of Evidence 

Again, this may be fairly obvious, but it is useful to review briefly what we mean by 
evidence. While some might interpret the term broadly to include such things as 
advocacy (evidence of which might be the number of calls to open line radio shows), the 
meaning I attach to evidence is a scientific one. The word empirical captures the tone. 
Empirical refers to a cluster of ways of gathering evidence through systematic 
observation, conducted under established “rules of evidence” long established in science. 



Experimental and Naturalistic Studies 

Broadly speaking, empirical evidence in science may be classified as either experimental 
or naturalistic. Experimental evidence is that conducted under highly controlled 
laboratory conditions (think of the inclined planes and levers of the ancient physicists or 
the high school physics laboratory). Naturalistic evidence is everything else, but 
essentially refers to observations conducted in field settings, under uncontrolled 
conditions (think of the field notes of Charles Darwin). More specifically, the following 
are some of the main characteristic of empirical evidence. 

Results derived from studies conducted under scientifically credible conditions, including: 

• Clear research questions or hypotheses from theory, previous research or practice 

• Clear definitions of variables 

• Clear observational/measurement techniques 

• Capable of replication 

• Error estimates 

• Peer review 
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Corresponding to the distinction between the experimental and the naturalistic, two main 
types of empirical scientific studies may be identified. In the human sciences, and in 
fields such as agriculture, experiments are now often referred to as randomized clinical 
trials (RCTs) because the essential control mechanism, in a situation where a large 
number of variables can influence the outcome of interest, is random assignment of 
participants to treatment groups. I will discuss the reasons for this a bit later. 

The human sciences counterpart to the naturalistic field study in the natural sciences is 
the survey. Surveys take many forms, from public opinion polls to large scale testing 
programs, but are characterized by the collection of a large amount of information from 
(usually) a relatively large sample of individuals in natural settings such as homes, 
schools, workplaces or medical clinics. 

It is crucial to stress that, no matter what the research field, it is rare to find that a single 
study, whether experimental or naturalistic, yields definitive results. It is in the nature of 
science that our studies are subject to design limitations, errors of measurement, and other 
flaws that require us to be skeptical of any one result. Nowhere is this more evident than in 
medicine, where it seems that almost every day a new study appears which seems to 
contradict previous research. The reality is that evidence can be considered definitive or 
useful only if it can be repeated under similar conditions and if it can hold to scrutiny from 
independent agents, more specifically others in the field who are capable of conducting the 
same kind of work. This introduces the ideas of peer review and synthesis, as essential 
elements in the accumulation of evidence. Indeed, synthesis of research in important areas 
has become common enough and well enough defined to qualify as a research form in its 
own right. Indeed, I would go as far as to argue that policy makers and practitioners need, 
and ought, to rely more on syntheses than on the original research, both for practical reasons 
- busy people do not have time to read and assess all of the research, and for more scientific 
reasons - synthesis research is deliberately designed to find patterns and expose flaws. 

It is important to note that many other research genres and methods exist. In particular, 
there is a strong movement in education towards the conduct of qualitative studies, a 
variation on naturalistic studies in which the data consist of detailed descriptions of single 
cases or a small sample. The archetypical qualitative study is the ethnography or detailed 
case study of a culture. I can think of a number of excellent examples of ethnographies 
involving the “culture of the classroom” including Smith and Geoffrey’s The 
Complexities of an Urban Classroom (1968) and Philip Cusick’s Inside High School 
(1973). However, these are few and far between, as evidenced by the lack of comparable 
more recent works. In my view, such studies have a place in helping to develop an 
understanding of a phenomenon at a theoretical level or in a way which helps identify or 
clarify research questions or hypotheses. Michael Connelly, a colleague well known for 
conducting high quality qualitative research once said to me, “It takes only one case to 
develop an insight.” In the end, however, the evidence required for policy and practice 
needs to be more generalizable than single case insights. The insight needs not only to be 
generated but also to be broadly tested. Despite the proliferation of qualitative studies in 
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our field, and their value in adding to our insights about teaching and learning, it is rare to 
see a policy decision influenced in any significant way by qualitative research. When 
policy makers speak of the need for evidence, the reference point is to “what works” in 
some general sense, and hence there is almost always a need for quantitative estimates. 



Randomized Clinical Trials: The Gold Standard 

It should be obvious that the study of teaching and learning (and research in the human 
sciences generally) is complicated by the fact that educational outcomes are influenced 
by a host of factors, not all of which are within the control of teachers or policy makers. 
We need only to refer to the influence of home and family circumstances (captured in the 
notion of socioeconomic status) to recognize this. The independent effects of any single 
factor can only be studied by isolating that factor from all others. There are two ways of 
doing this - experimental and statistical. Properly designed experimental studies can 
definitively achieve the desired degree of isolation (however, usually at the price of 
generalizability, as we shall see). Statistical control is widely used but is, by definition, 
much less effective because it is impossible to anticipate all of the factors that might need 
to be controlled. 

How can a host of factors, related to outcomes but not of direct interest in a specific 
study, be controlled simultaneously? In all of my years of teaching research design to 
graduate students, I have found that one of the most difficult concepts to understand is 
that of how random assignment to treatment conditions (usually called experimental and 
control but other combinations are possible) can control for all variables other than the 
one under study. Simply put, in adopting random assignment to treatments, we simply 
give up on the impossible task of identifying all of the relevant variables and controlling 
them one by one, and rely instead on the well established fact that, for any reasonably 
large sample (and it does not have to be very large), random assignment of individuals to 
treatment groups ensures that those in each group will be relatively evenly divided on all 
factors which might contribute to an outcome, except for the one of interest in the study 
at hand. The key is not in keeping everything constant but in balancing out the 
extraneous variables across groups. Furthermore, it is also possible to use the laws of 
probability to compute the accuracy of the attempt to balance. This also happens to 
depend only on the sample size. 

To take a simple example, if we had 20 students, 10 boys and 10 girls, to assign to 
treatment and control groups, we could proceed to assign in a matching sense, to ensure 
that we had exactly five of each in each group. We could also randomly assign, and hope 
to achieve the same result. Resorting to common sense, just for a moment, it is not 
difficult to imagine that it would be harder to achieve this result with randomization than 
with matched assignments. However, if the students also vary on five or six other 
variables such as age, ability, home background, and so on, the matching task becomes 
much more difficult. If we do not know exactly which variables are relevant for 
matching, the matching task becomes impossible even in this simple case. 
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Unfortunately, with such a small sample, randomization will not work very well either. 
However, if we had 2,000 students, evenly divided between boys and girls, the chances 
would be very good that we would achieve a close gender balance by random assignment, 
without having to rely explicitly on gender matching. (A gender difference of 10-20, 
which would be typical of random samples of this size, would not matter very much in a 
sample as large as this). Extending this analogy, it can also be said that all possible 
variables that might interfere with the outcome would balance out with the same degree of 
accuracy. Hence, the randomization process works for any situation, without the added 
complexity of trying to figure out which variables to control and with the added bonus of 
knowing the degree of accuracy involved. 



Examples from Medicine 

In looking at areas within the human sciences where the RCT model is being applied, 
there is no doubt that medicine stands out both in the number of studies of this nature and 
in the results achieved by adopting RCT as its core method for determining what works. I 
therefore thought it useful to give a couple of examples of where this type of research has 
had a very large impact. I will leave to later any comments on the appropriateness of 
transferring a medical research model to education. 

The two medical examples I want to use are different in many ways but share the one 
feature of use of RCTs. You may have heard of the Hormone Replacement Therapy 
(HRT) study as it is fairly recent, high profile, and highly relevant to a large population of 
women. The work of the Children’s Oncology Group (COG) is less well known, not at 
all splashy, and rarely makes the news, but this is an excellent example of how persistent 
plodding work, conducted over an extended period, can yield profound results. In my 
view, it is also a great example of how we might consider organizing research in 
education so as to make it more effective and useful. 

The Children’s Oncology Group is a large consortium of teaching hospitals in the United 
States and Canada dedicated to research on childhood cancer. Although its incidence is 
relatively low among children, cancer remains the second leading cause of death in 
children (after accidents) and tends to be regarded as a particularly devastating disease, 
with a very high public profile, as evidenced by the extent of fund-raising that has been 
linked to this disease. Until the early 1970s, most forms of childhood cancer had much 
less than 50% survival rate and no effective treatments were available. 

Beginning in the 1970s, the predecessor organizations of COG began to tackle the question 
of treatment, and particularly that of how the emerging new field of chemotherapy could be 
applied to childhood cancers. The work has consisted of a long series of RCTs, with each 
new experimental treatment representing an increment of change (usually an increase in 
intensity of chemo or variation in the mix or sequence of drug administration) over the 
previously accepted treatment. In each case, children were randomly assigned, with 
consent of parents, to either an experimental or control treatment, where the control was 
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always the standard treatment being used at the time (obviously children could not ethically 
be deprived of treatment). If the experimental treatment proved to yield even a small 
increment of progress (where progress was measured by survival), that treatment became 
the new standard and the cycle was repeated. 

Figure 14.1 shows the relative survival rates for several varieties of childhood cancer 
over about a 25 year period. Substantial improvement is apparent in all varieties, with 
the largest improvements being in those diseases which had the lowest survival rates at 
the beginning of the period. Figure 14.2 shows a breakdown of survival rates by specific 
disease. While progress is somewhat uneven, the trend is upward in all cases, and 
dramatically so in the most severe form of leukemia, known as AML. 

There is no doubt that almost all of the improvement in survival has been a consequence 
of the systematic use of RCTs to investigate the effectiveness of treatments. These 
survival rates are independent of changes in incidence of the disease, socioeconomic 
factors, alternative treatments (which some children may undergo outside the studies) or 
other extraneous factors, simply because all such factors have been balanced out in the 
treatment designs. While any one experiment may yield only a small increment in 
improvement (and some yield adverse effects), the cumulative effect is large and positive. 

The Hormone Replacement Therapy (Rossouw et al., 2002) study differs significantly 
from the COG studies in focusing on a single syndrome and being a single, relatively 
short term (by COG standards) very large sample study. It is also one of these rare cases 
where a single study has had a dramatic effect on treatment even though many would 
argue this effect was not warranted by the results. 
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Figure 14.1. Cure Level of Children and Adolescents with Cancer 
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Figure 14.2. Cancers in Children < 15 Years of Age: Trends in 5-year Survival Rates 



The key hypothesis of the HRT study was that HRT will reduce heart disease and 
fractures without increasing breast cancer. The study used a sample of more than 16,000 
women, aged 50-79 (that sample is part of a much larger women’s health study). Women 
were randomly assigned to receive either hormone replacement therapy or a placebo. 

Data were collected for 8-12 years, an example of a time frame that is almost unheard of 
in education studies. 

In this case, interestingly enough, the control was no treatment at all, as no alternative to 
HRT was available. In this case, unlike COG, which must rely on a large consortium to 
find enough cases to yield reasonable sample size, the target group, namely women over 
50, is not hard to find, nor is there any shortage of seekers of treatment for the symptoms 
HRT was intended to prevent. In this case, it is important to point out that the research 
question had nothing to do with the efficacy of the treatment in reducing the symptoms of 
menopause. That had already been established through the earlier RCTs required as part 
of the drug approval process (another interesting contrast with how education treatments 
are adopted). 

Since this HRT treatment is not particularly difficult to administer (blindly to both 
patients and their doctors) and is not concerned with a life or death matter, it was 
obviously possible to recruit a very large sample and to obtain consent for randomization. 
(It is important to note that consent is obtained not only to participate but also to be 
randomly assigned. Patients do not get to select their treatment in RCT). 
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Figure 14.3 shows the results of the HRT study. It is interesting to note that effects other 
than the hypothesized ones were also investigated. It is clear that most of the effects 
were not favorable to the therapy. Only hip fractures and colorectal cancer showed 
positive effects for the treatment, while most of the higher incidence events showed more 
positive effects for the placebo group. 

The fallout from this study was dramatic. First, the study was actually stopped in 2002 
before its scheduled completion because the authors considered the adverse effects to be 
too dramatic to allow them ethically to continue to administer the treatment. Second, the 
study made headlines around the world and, by all accounts led literally millions of 
women to stop HRT treatment or scramble to see their doctors on what to do. I have not 
been able to find evidence on the effects of this study on sales of Premarin and related 
drugs because pharmaceutical companies are not particularly forthcoming with such 
information. 

It is interesting to note that, though severe, the adverse events were relatively low 
incidence, raising the question of whether the established benefits for the condition 
actually being treated outweigh the small increase in risk of cancers. The debate over this 
trade-off continues, illustrating another point about evidence. No matter how compelling 
the evidence, research cannot tell us what we should do. Evidence can only inform us of 
the consequences of doing certain things. The goal of research is to obtain the best 
evidence, not to actually make the decision. Professional judgment is not displaced but is 
enhanced by the evidence. 




Figure 14.3. Disease Rates for Women on Estrogen Plus Progestin or Placebo 
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Education Examples 



RCTs in education are much fewer and further between. After a bit of a flurry in the 
1960s, mostly consisting of small scale experiments on teaching methods (remember 
discovery learning versus structured teaching?), most of which yielded no significant 
differences, RCTs seemed to fall into disuse (if not disrespect) until the 1990s. With 
increased concern for accountability, accompanied by financial constraints, came 
demands for evidence to support what we are doing in education. The most striking 
immediate response to this was not to restore interest in RCTs but rather for governments 
to support large scale surveys. However, a few examples can be found of RCTs which 
seem to have had substantial effects on policy. I will refer to only two of these, the 
Tennessee STAR class size experiment and the High Scope/Perry Preschool Program. 

Going back to our earlier example of class size, the Tennessee STAR (Student Teacher 
Achievement Ratio) study (Finn & Achilles, 1999) was a large scale RCT that began in 
1985. This has been described by many commentators as one of the most significant 
studies in education for the past 25 years. In this study, more than 1 1,000 grades K-3 
students and their teachers in 79 schools were randomly assigned to three treatment 
conditions: class size 13-17, 22-26, and 22-26 with a teaching assistant. Data on a large 
number of background variables and on achievement outcomes were gathered and 
students were followed as far as grade 10. 

After four years, the achievement results clearly favored the small class size group. 

There were no differences between the larger classes with and without the teaching 
assistant. Class size effects were greater for minority than for majority students. 
Although the experiment was discontinued after grade 3, lasting achievement effects 
were found up to grade 10. Follow-up studies up to 1999 indicated that students in the 
small classes had better high school graduation rates, higher grade-point averages, and 
were more inclined to pursue higher education. Some projections from the study have 
even gone as far as to argue that the costs of the small classes will be more than offset by 
better labor market performance of the small class group. 

Tennessee STAR and a couple of other similar large scale RCTs on class size have been 
widely used to justify policies on limiting class size in the early grades in many 
jurisdictions, including a major initiative here in Alberta. This is a clear case where the 
evidence supports intuitive and advocacy positions, although cost remains a major 
deterrent to creating classes small enough to make a measurable difference. Also, as we 
shall see, the class size evidence is much less clear at later grade levels. 

As a second example, I have chosen a much smaller scale but equally famous study 
known as the High/Scope Perry Preschool Program (Schweinhart et al., 2005). That 
study dates back to 1962 but remains current because participants have been followed 
ever since and the study continues to be widely cited as evidence of the efficacy of early 
childhood literacy programs. In that study, 123 disadvantaged African-American 
students, ages 3 and 4, in Ypsilanti, Michigan were randomly assigned to an intensive 
early literacy program and a no-treatment control group. Early results showed substantial 
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gains for the treatment group on a variety of literacy measures. However, the main 
attraction of that study is the long-term, follow-up component. Figure 14.4 shows some 
results from a follow-up in 2000 when the participants were around age 40. As can be 
seen, all of the outcome measures continue to favor the experimental group. 

As in Tennessee STAR, some authors have argued that this and similar studies show that 
the payoff from intensive early childhood education programs vastly exceeds the initial 
costs. In this case, a small number of such studies seem to be sufficient to convince 
policy makers that a large scale move to pre-kindergarten programs is required, and many 
jurisdictions (though, as I understand it, not Alberta) have begun to move strongly in that 
direction. Again, although the evidence in this case is sparse, its direction is consistent 
with intuitive and advocacy positions. However, in this case, both cost and ideology 
(whether centre-based care is preferable to home-based care) are deterrents. 



Source: Schweinhart, 
et. al, 2005 

Arrested 5 times by age 40 



Earned >$20K at age 40 



Graduated high school 



Achieved basic outcomes at age 14 



IQ >90 at age 5 



0 20 40 60 80 100 



■ Program 
- No Program 




Figure 14.4. Perry Preschool Program Results at Age 40 



Criteria for Assessing the Quality ofRCTs 

Some of the main requirements for a high quality RCT are perhaps clear from the 
examples. More precisely, the following criteria may be used to judge the quality of 
RCTs: 

• Representative selection from an appropriate population 

• Random assignment to treatment groups 
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• Fidelity of implementation of the treatments 

• Quality of measurement 

• Long-term follow-up for lasting effects 

• To what other populations can the results be generalized? 

The key requirement is random assignment to treatments. Anything else (matching, before 
after, intact groups) is a compromise which weakens the design. I stress sample 
representatives because there seems to be a mistaken view in many quarters that large 
samples are needed for generalization of results. In reality, sample size is not the key 
element. Small samples can give relatively accurate results (remember that we can 
compute the level of accuracy). The real issue is how well the population from which the 
population is drawn is representative of some broader population. This is the main flaw 
with the Perry study and many other similar studies in education. Disadvantaged black 
children in a medium-sized American city are a relatively narrow group. Yet this study has 
often been used to justify intensive preschool programs for quite different populations. We 
might debate the generalizability of even a large study such as Tennessee STAR because of 
the narrow range of grade levels used. However, all of this can be solved by replication of 
studies across broader samples. 



Surveys 

Unfortunately, though fairly easy to design, RCTs are difficult to implement and are not 
likely to be feasible in every area that might be of policy interest. This brings us to the 
other, much more common policy research type, the survey. While the emphasis here will 
be on large scale analytical surveys, smaller scale surveys are in common use for a 
variety of purposes, ranging from public opinion gathering to program evaluation. If the 
RCT is considered the gold standard for policy research, the silver medal must go to the 
survey. Although some would argue that the value of surveys relative to RCTs is perhaps 
comparable to the price of silver compared to gold, that is not my position. My own 
sense is that, in the near future, we are more likely to be able to implement good large scale 
surveys than good large scale experiments. Indeed, that is already being demonstrated. 

The fundamental difference between the RCT and the survey is the absence of random 
assignment to treatments. Group comparisons from surveys must be made using whatever 
natural groupings are found. For example, a class size survey would look at class size 
differences as they occur naturally in schools. Under these conditions, class size is likely 
to be confounded with many other variables that are relevant to outcomes, such as school 
size, socioeconomic status, quality of teaching, and many others. Set against this major 
drawback is the fact that surveys are easier to implement, especially on a large scale, 
surveys make it possible to look at many effects at the same time, and or particular 
interest in many policy areas, surveys can provide descriptive information on a system as 
well as comparisons across systems. 
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It is useful to make a quick distinction between surveys and population studies. Much of 
the core data that are useful for policy making can be gathered using data that are already 
collected in support of the routine running of the system. Examples would be enrolment 
counts and projections, costs, or achievement levels. Most jurisdictions have now 
become fairly sophisticated at this kind of data collection and data on the full population 
(referred to generically as census data) may sometimes be used to good effect in 
addressing research questions. 

The problem is that we do not have census data on very many things. The sample survey 
is a means of collecting much more data in an economically efficient way by applying the 
measures of interest to a sample instead of to the entire population. As for RCTs, under 
appropriate sampling conditions, we can estimate the error associated with having only a 
sample instead of the whole population. In fact, samples do not have to be very large to 
provide highly accurate estimates, as illustrated by the closeness of political opinion poll 
results to election results even though polls rarely sample more than 1,000 or so 
respondents out of millions of voters. 

The inability to experimentally control extraneous variables can be offset to a significant 
degree by using statistical controls. For example, if we think that class size influences 
achievement but that class size itself is influenced by school size, techniques exist for 
controlling school size and providing estimates of the class size effect that are independent 
of school size. While statistical control is not a perfect substitute for random assignment, 
its use allows us to capitalize on the many advantages of surveys while offsetting the main 
disadvantage. 

I don’t need to give medical examples to illustrate well designed large scale surveys. I 
would note, however, that surveys are analogous to what are called epidemiological 
studies in medicine. For example, the first studies of the link between smoking and lung 
cancer came from the large scale collection of very simple data which compared the 
incidence of lung cancer in smokers and non-smokers. In this case, conducting a RCT 
would have required that a random sample of participants be persuaded to smoke and a 
second random sample persuaded to refrain from smoking and both would have had to be 
followed for 30 years or so before the effects became evident. This well illustrates how 
RCTs can break down on both ethical and feasibility grounds. Nevertheless, the results 
of the early epidemiological studies were persuasive enough to induce the US Surgeon 
General to issue a warning in 1962 about the health hazards of smoking. Although, 40 
years later, smoking is far from being eradicated, I think we would all recognize that 
survey research has made a major contribution to policies around smoking and to its 
incidence. 
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Large Scale Survey Examples 

Fortunately, I can give some survey examples that are a bit closer to home, because 
Canadian work is available and because I seem to spend a lot of my time working with 
studies of this nature. In the interest of time, I will simply draw a small number of results 
from two large scale Canadian studies, the Student Achievement Indicators Study (SAIP) 
and its successor the Pan-Canadian Assessment Program (PCAP) and one international 
study, the OECD Programme for International Student Assessment (PISA), and 
specifically the Canadian component of that study. 

In these studies, the target populations are national, and in Canada, provincial in scope. 

The studies are aimed at students from age 13 to age 16. The main goal of these studies 
is comparative - specifically the comparison of achievement levels in core subjects 
(reading, mathematics, and science) across jurisdictions. These studies are conducted 
under the most sophisticated sampling designs available, allowing national and provincial 
estimates of achievement to be made with error rates of around 2%. 

The comparative results from these studies have been fairly widely circulated and you 
may have seen some of them over the years. Figures 14.5 to 14.7 give some examples of 
results for Canada. 

The different colors on the bars in the charts represent clusters of provinces that are 
significantly different from each other. Provinces with the same color are not significantly 
different. However, given that we are in Alberta, I would be remiss if I did not point out 
that many surveys of this nature have shown Alberta to have the highest scores in Canada 
and often close to the highest in the world. If this is the measure of Alberta’s aim to have 
the best K-12 system in the world, then the international results indicate that you are close to 
meeting the goal. On the other hand, the most recent PCAP results present a different 
picture, with Alberta 13-year-olds significantly lagging those in Ontario and Quebec. 

Again, I caution here that one study is not necessarily an indication of a trend, though I 
suspect that this latest result is causing some consternation in some quarters in this 
province. 

The details of these results are less important for our purposes than the idea that 
comparisons of this nature can be made with some accuracy from sample surveys. More 
important for the issue at hand is whether we can make any inferences from these results 
about factors that contribute to higher or lower achievement. Time does not allow many 
detailed results to be presented. I will simply summarize a few of the most interesting 
findings. 
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Figure 14.5. Canada Results, PISA 2000 Reading 




Figure 14.6. Canada Results, PISA 2006 Science 
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PCAP Reading 2007 
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Figure 14.7. Canada Results, PCAP 2007 Reading 



First, back to class size. SAIP and PISA consistently give results which are in the opposite 
direction to those of Tennessee STAR and other class size experiments. Keeping in mind 
what we have said about confounding of variables, class size is related to a number of other 
factors (school size, urban/rural location, socioeconomic status) which themselves are 
correlated with achievement. Hence, it is possible that the class size effect is suppressed or 
offset by these other effects. This is where the notion of statistical control comes in. 
Without getting into the details, it has been found that, even after controlling for other 
relevant factors, the class size effect found in these studies tends to be neutral or favor 
larger classes. Again, without going into a lengthy explanation of what is happening here, 

I note only that the grade levels assessed by SAIP and PISA are different from those in the 
RCT studies and that other studies of junior high school class sizes are consistent with 
SAIP and PISA and not with Tennessee STAR. I also suspect that in many countries, 
lower ability students tend to be assigned to smaller classes to allow for more 
individualized instruction. Finally, as we have seen, there is a trade-off here between the 
limitations of surveys in allowing causal inferences and the advantages of scale and 
generalizability that surveys yield. 

Figure 14.8 presents some highlights of other results from these large scale surveys. 
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Some Factors Positively Related to Achievement 

• Longer class periods 

• Fewer lost school days 

• More homework 

• Greater teacher specialization 

• Positive school climate 

• Positive classroom disciplinary climate 

• Direct instruction (giving notes, showing how to solve problems, teacher asks 
questions) 

Some Factors Negatively Related to Achievement 

• Being tutored in mathematics 

• Absence from school 

• Time lost due to disruptions 

• Hours watching television 

• Working in groups in the classroom 

• Doing projects in mathematics 

• Using computers, software, AV resources 



Figure 14.8. Factors Related to Achievement From These Large Scale Surveys 

I won’t go over all of these but some seem to confirm our intuitive or common sense 
notions whereas others, like the class size results, are counterintuitive. Also, it is 
interesting to note that some of the more positive effects are in areas that are in the realm 
of practice, and are under the control of schools and teachers. Implementing some of 
these positive practices does not have to cost much nor do they require major new policy 
initiatives. Indeed, reviews of research on student achievement over the years tend to 
reveal that school and teacher level processes are likely to have more effect than policies 
developed at higher levels such as districts or provinces, something which fits well with 
the school improvement model on which AISI is built. 



Criteria for Assessing the Quality of Surveys 

Most of the criteria for assessing the quality of surveys are similar to those for RCTs. 
The main ones are: 

• Representative selection of a sample from a relevant population 

• Rationale for selection of variables to be measured (because of the large number of 
variables in a typical survey) 

• Quality of measurement 

• Quality of statistical controls 



AISI Colloquium 



115 



Large Scale Improvement 



In the case of surveys, the key element is population representativeness. Again 
randomization is the key to representativeness. However, in this case, randomization 
applies only to sampling from the population of interest, not assignment to treatments. 
Large samples for a given population are generally not required. For example, even for 
the largest populations, a random sample of 1,000 or so yields relatively small error. Of 
course, large scale surveys are often concerned with many different populations (for 
example covering the 20 or so Canadian populations for SAIP or the 60 or so PISA 
countries, requires very large total samples). As is the case for RCTs, the quality of 
measurement is important and having a basis for selection of variables to be included in 
the survey looms larger. Statistical control is crucial if we want to get as close as 
possible to causal inferences. This is one area in which technical advances in recent 
years have dramatically improved the tools available. The problem, as I see every day in 
my own work, is that some of the statistical techniques now available, such as 
hierarchical or structural equation modeling are likely to be viewed by teachers and 
policy makers as something akin to black magic. Simple comparisons of means across 
treatment groups in RCTs are obviously much easier to make sense of than regression 
coefficients. 



Quasi-Experiments and Other Methods 

Before leaving the topic of research designs, it is worth mentioning that variations on pure 
randomized clinical trials, often called quasi-experiments, are often used in education. 
Examples are pretest-posttest studies, without control groups, where the inference is that 
change over time is evidence of treatment effectiveness. Of course, the fundamental flaw 
in this reasoning is immediately evident. Such a study cannot answer the question of 
whether the change would have occurred in the absence of the treatment. However, 
modifications of this design may be used to good effect. For example repeatedly applying 
and withdrawing the treatment and observing the trend, or collecting data over a long time 
prior to the treatment, immediately following the treatment and for some time after, can 
strengthen any inference that the treatment is the cause of the effect. Similarly, while 
strict random assignment of individuals to treatments may not be possible, random 
assignment of groups, such as classrooms, may be possible. 

A variation on the survey is the population study. A common form of this study is the 
use of data collected on a full population, often for purposes other than that of the study. 
Examples are the use of census data to study relationships between geographical location 
and learning or the use of provincial assessment data to examine class size. Large scale 
administrative data bases, often accumulated for simple purposes such as tracking 
enrolments or costs, can often be turned to more analytical uses. Indeed, exploiting such 
data bases, called data mining in business, has become a large industry in itself. Much of 
the data collected by banks, insurance companies, credit card agencies, and even 
governments are now being amassed for use in such areas as customer profiling, targeted 
marketing, and a host of other applications, some desirable, some not. In education, we 
have just begun to exploit the power of data mining to shed light on policy and practice. 
Although the data are not always tailored to the purposes of a specific study, data mining 
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is an inexpensive way to conduct analytic work because the large expense of collecting 
the data has already been borne by someone else. While privacy legislation limits the use 
of individually identifiable records, the same legislation also permits research that is in 
the public interest to be conducted using public data bases. 



The Importance of Synthesis 

Synthesis was mentioned at the beginning as a type of research in its own right. Indeed, 
specific synthesis techniques such as meta-analysis, best evidence synthesis, and the review 
methods of organizations such as the Evidence for Policy and Practice Information 
Coordinating (EPPI) Centre in the United Kingdom and the Campbell Collaboration in the 
United States are now widely used to help bring together the available evidence on 
particular topics. Actually, the term “knowledge mobilization” is now being used to reflect 
a relatively new trend towards deliberate attempts not only to conduct syntheses, but to 
make these syntheses more readily available to users. The emphasis on knowledge 
mobilization is based on the idea that ready access to research is crucial but that policy 
makers and practitioners rarely have time to review individual research studies. 

In a crucial way, reliance on research synthesis is a form of reliance on expert opinion or 
authority, as indicated earlier. While there is good reason to be skeptical of expert 
sources, especially those with vested interests, there is consolation in the fact that 
research synthesizers and knowledge mobilization organizations operate on a scientific 
basis and not an advocacy principle. In particular, their methods and sources are 
transparent and themselves subject to peer review. 

Let me give a couple of quick examples of syntheses that appear to me to be good enough 
to be used as evidence for policy. 

The Canadian Centre for Knowledge Mobilization (CCKM) is based at the University of 
Waterloo. Its mandate is to conduct systematic reviews on topics in education that are 
judged to be of policy significance. CCKM has recently completed a study of early 
childhood education programming, timely in light of the controversy over whether 
children are better cared for at home or in outside settings. The following are some key 
elements of this synthesis: 

• 66 studies with more than 500 measures of outcomes were reviewed. 

• Centre-based child care settings yield more positive outcomes than home-based 
settings. 

• Time in care and intensity of programs are associated with more positive outcomes. 

• Higher quality care, as rated by a commonly used scale of quality, is associated with 
improved development. 

• Benefits are greater for disadvantaged children. 

• Higher levels of teacher training yield improved outcomes. 

• Adult-child ratios generally yield no significant effects. (Canadian Centre for 
Knowledge Mobilization, 2006) 
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In general, these results clearly favor centre-based over home-based care and more 
intensive over less intensive programs. However, the results also show that, within the 
centre-based situation some characteristics make a difference and some do not. These 
results are consistent with several other syntheses, all of which make a strong case for the 
benefits of child care centres. These reviews come close to settling the argument for 
establishing a stronger child care system than we now have in Canada. However, I 
hasten to add that they do not settle the ideological debate in this area or the issue of who 
pays for the system. The evidence strengthens one side of the argument over the other 
but does not necessarily bridge the ideological divide. 

A second example is a Campbell Collaboration Centre review of peer tutoring (Ritter et 
al., 2006). This refers to the use of older students to tutor younger ones. Peer tutoring has 
considerable appeal as a relatively simple and inexpensive way to give extra help to 
students needing it, with possible benefits to both the tutor and the tutee. This is a great 
example of a practice innovation, which is well within the power of schools to implement, 
without much input from any broader policy level. 

The Campbell Collaboration and other knowledge mobilization organizations tend strongly 
to favor RCTs over other forms of research, and sometimes confine their reviews to studies 
of this nature. In the case of peer tutoring, this was the case because naturalistic studies of 
peer tutoring would be fraught with so many control problems as to be virtually useless. A 
total of 28 RCTs were found covering peer tutoring in reading, writing, and mathematics 
outcomes. The review found positive results for overall reading, reading letters and words, 
and oral fluency but no significant effect for reading comprehension and mathematics. 

Again, since peer tutoring can be implemented at little or no cost, evidence of this nature 
could be used fairly directly as a basis for implementing a peer tutoring program in a 
school and for determining the areas in which this might be effective. 



Credibility and Uses of Evidence 

The examples I have given all involve studies that are well designed and mostly on a 
large enough scale to allow us to say something about broad populations. However, we 
all know that thousands of research studies are published in education every year and that 
most do not meet the standards set by the examples given. That is equally true in all 
other fields, including medicine, a point that has been made forcefully by the long-time 
editor of the British Medical Journal Richard Smith, who is on record (Smith, 2007) that 
most articles in medical journals are virtually worthless by themselves. All evidence is 
clearly not equal. What makes some evidence more credible than other? While some 
specific criteria for assessing the quality of studies have already been given, a couple of 
broader comments might be useful. 

First, all things being equal, RCT evidence is much stronger than survey evidence. A 
well designed RCT can isolate a variable and permit a claim that that variable is the cause 
of the outcome of interest. However, all things are seldom equal. The intensity, 
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disruption, and cost of RCTs mean that they must be used selectively and typically on a 
small scale. Scale is where surveys come into their own. It is entirely feasible to conduct 
surveys on provincial, national, and international scales and to include in surveys many 
different factors that might be related to the desired outcome. 

This trade-off between causal inference and scale is at the heart of the difference between 
RCTs and surveys and is crucial to the assessment and use of evidence. This trade-off 
corresponds to the traditional distinction that all graduate students are (or perhaps were) 
taught between internal and external validity. Internal validity refers to the ability to 
attribute the outcome to the treatment, the essence of the RCT design. External validity 
refers to the ability to generalize the result to a broad population. While RCTs can 
possess this attribute, this requires a reasonably large scale study. Except in a situation 
such as the COG consortium, where it is possible to select a sample representing a very 
wide area (e.g., all childhood cancer patients in North America), broadly representative 
RCTs are difficult to design. This is especially true in education where jurisdictional 
boundaries are strong and the approval process for intervention studies complex. For 
example, it is difficult to imagine a study such as Tennessee STAR being able to cut 
across jurisdictional boundaries. While it might be reasonable to generalize a study 
conducted in one jurisdiction to others, this is hazardous. For example, class sizes in 
some jurisdictions may already be as small as the smallest ones used in Tennessee STAR, 
so the small class treatment may not amount to any change at all. 

Well designed, large scale surveys are, on the one hand, quite feasible and can be said to 
possess much stronger external validity than RCTs both because of sample scope and 
because the measures are made under real world conditions. Assuming that appropriate 
statistical controls are applied, it can be more plausible to make policy inferences from 
survey data than from RCT data because surveys can be more generalizable. On the 
other hand, replication of small scale RCTs can help solve the generalizability problem. 
The essential trade-off thus becomes one between scale and replication. 

To the user, then, the key point is that evidence needs to be judged on the grounds of both 
internal and external validity. Small, highly internally valid RCT studies are likely to be 
useful only if they have been replicated. Large, externally valid studies, for all their 
internal validity weakness, can be used with more confidence as single studies, but only if 
appropriate statistical controls have been used to help isolate the effects of interest. In 
particular, it is possible to place great confidence in comparative results (such as mean 
scores across provinces or countries) as estimates of population values, especially where 
the margins of error have been identified. However, much less confidence can be placed 
in attempts to identify the causes of such difference typically because of insufficient 
control of differences in characteristics of the populations being compared. 
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Conclusions and Implications for AISI 



Evidence and Intuition 

A comment is needed on what seems to have been interpreted as a strong contrast 
between my emphasis on evidence and the message conveyed in Andy Hargreaves’ 
presentation on the need for a balance between evidence and intuition. Thinking about 
this, it seems to me that the difference is not as great as it might appear. Andy’s plea was 
not to devalue the role of intuition but to ensure a balance. Because I see the current 
policy making environment as strongly tilted towards intuitive decisions (which for now 
is taken to include all of the other influences except evidence), my plea is also for greater 
balance. It is possible that I feel that we have further to go on the evidence side than does 
Andy, but that is a matter of detail. My plea is not that we move to uncritical empiricism 
or to ignore all influences other than empirical evidence. In a political system, that is 
impossible. There are lots of areas in which the evidence simply does not exist and we 
cannot bring the system to a halt while awaiting the evidence. Nevertheless, compared to 
where we are now, I simply see the need for a substantial shift towards the use of 
evidence. While agreeing that we must often act intuitively, I would also argue that, 
where there is conflict between intuition and evidence, we should never act in ways that 
are contrary to evidence. 

Details aside, I do have a concern over a tendency that I continue to see in education to 
dismiss evidence on the grounds that children are unique beings that teachers can judge 
better than anyone else what is good for their students, and that judgements on teaching 
practice are best left to teachers. While a case can be made that teachers should enjoy the 
same level of autonomy as other professionals, we need to remind ourselves that 
professional autonomy is not the same as a license to do anything one wants. Virtually all 
professions, much more than teaching, operate under evidence-based standards of 
practice that individual practitioners ignore at their peril. 

Without wishing to place too negative a tone on the argument, virtually all professionals 
except teachers are subject to disciplinary codes and malpractice litigation if they fail to 
maintain an adequate standard of care in their work. Even in architecture, where creative 
license is so highly valued, leaky roofs are not tolerated, even in the most innovative 
building designs. In medicine, where closing the examining room door is akin to closing 
the classroom door, physicians can, and often are, quickly taken to task if their practice 
falls outside of established ways of diagnosing and treating their patients. While not all 
of medical practice is as strongly evidence-based as it might be, even the most basic 
practices, such as hand washing, have a clear evidentiary basis. Indeed, the current 
campaign to reinforce the importance of hand washing has its origins in recent studies on 
this and other elementary infection-control procedures. 
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What is AISI? 



I need to say at the outset that I believe AISI to be an impressive, exemplary venture that 
has few parallels in the world of school improvement. AISI represents one of the few 
large scale school improvement programs to be well documented and subjected to 
systematic review. The attempt to synthesize results across projects and over time is 
entirely consistent with the spirit of evidence-based decision making that I have 
attempted to convey. There are many lessons to be learned from AISI and the program, 
though widely known, deserves even greater attention than it has received. While we 
cannot attribute the high performance of Alberta students to AISI in any causal way, AISI 
may well be representative of a spirit of innovation, cooperation among partners, 
attention to outcomes, and a focus on performance that seems to characterize Alberta 
education in general. 

AISI can be looked at in many ways. However, from the perspective of this presentation, 
AISI can be seen in one of two ways: either as a series of action research projects for 
individual schools and districts, with no intent to draw any conclusions for anyone else in 
the world, or as a set of quasi-experiments, designed to yield replicated results that may 
have value to others. More generally, most AISI projects involve treatments of some 
sort, and most focus on outcomes. In principle, therefore, AISI may be conceptualized as 
experimental in nature. 

Since our concern is with evidence that can yield generalizable results, it is useful to look 
at AISI projects as quasi-experiments. More specifically, AISI can be viewed as a set of 
before and after designs (not quite time series designs because there are no long-term 
before and after measures or no repeated applications or withdrawals of treatments). 
Taken individually, before and after designs are relatively weak, because there is no way 
to attribute the effect to the treatment. However, treated as replications, such designs can 
provide more compelling evidence of an effect if a pattern of results in the same direction 
can be found. 



AISI Synthesis 

This leads us to a second feature of the AISI reporting system - that of the provincial 
cycle reports as syntheses. Indeed, one of the most common synthesis techniques, known 
as meta-analysis, has been used in these reports. Without going into the details of meta- 
analysis, this technique provides a way of combining the effects of a large number of 
studies by computing a standardized measure called, naturally enough, the effect size and 
averaging these effect sizes. 

Figure 14.9 gives one example of how this was reported in Improving Student Learning: 
Provincial Report for Cycle 2 (Alberta Education AISI, 2008). The meta-analysis was 
conducted over two types of outcome measures, surveys and learning assessments, with 
teachers and students (and parents) the survey targets, and local and provincial assessments 
as the learning outcome measures. 
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Three Year Average Effect Size 
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Figure 14.9. Three Year Average Effect Size for AISI Projects, Cycle 2 46 



In interpreting this figure, it is noted that conventions have been adopted for effect sizes 
that may be considered negligible (less than 0.20), small (.20- .49), medium (.50-. 79) and 
large (.80 or higher). By this convention 47 three of the four types of measures yielded 
small effects, while the provincial assessment measures yielded negligible effects. 
Results based on the combined effects over the first two AISI cycles yielded slightly 
higher effect sizes but a similar pattern across the measures. The point here is not to 
debate the size of these effects but to illustrate the method of analysis. It is rare to find 
any work of this nature associated with school improvement programs, and its existence 
points clearly to a goal of AISI being to find generalizable results, at least for the 
province. 

The AISI reports yield a wealth of additional, more detailed, information. Time allows 
me to point only to one other example. Figure 14.10 shows changes in the nature of the 
projects supported under Cycle 2 compared to Cycle 1. The value of this chart for our 
purposes is that it illustrates more clearly the kinds of interventions which characterize 
AISI. Many of the more controversial areas of teaching and learning are represented 
here. However it is clear that in Cycle 2, there was a distinct shift generally towards 
teaching and learning projects towards four areas in particular: hands-on experiences, 
enrichment, differentiated instruction, and teaching to accommodate learning styles. All 
of these are important ways of teaching for which evidence on effectiveness is needed. 



46 Source: Alberta Education AISI (2008, p. 30). 

47 The AISI report uses a slightly different version: no effect (less than 0 or not significant), minimal (.01 
to less than .20), small (.20 to .30), medium (.04 to .07), large (.80 or higher). 
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5 10 15 20 25 30 



Hands-on experiences 
Enrichment 
Differentiated Instruction 
Teaching to Accommodate Learning.. 

Alternative Delivery 
Individual Programming 
Problem Solving 
One-on-one/Small Groups 
Teaming/Cooperative Learning 

■ Cycle 1 □ Cycle 2 



48 

Figure 14.10. Percent of AISI Projects by Teaching Strategy 




Again, because it is possible to synthesize over many projects, and to enhance 
generalizability by having projects in a variety of different settings (presumably in 
different kinds of schools and at different grade levels), evidence from AISI on these 
areas can potentially make a significant contribution to our overall knowledge of these 
teaching and learning strategies. 

There are two main limitations in the provincial reports to date. First, there is insufficient 
descriptive information on particular interventions to allow a treatment to be replicated. 
Second, there is insufficient differentiation in the synthesis data to allow us to determine 
the grade levels, subject areas, or specific outcome measures for which particular 
interventions are more or less effective. We need to know, for example, what a treatment 
such as hands-on experiences consists of and how much variation there was across 
applications of that treatment. We also need to know if such an intervention is equally 
effective across the board or if it is differentially effective by grade level, student 
characteristics or other factors. 



48 Source: Alberta Education AISI (2008, p. 13). 
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Lessons for Future Cycles 



In all, about 1,700 projects have been funded over the three AISI cycles to date and a 
fourth cycle is planned. In my experience, a documented data base of interventions on 
this scale is not available anywhere else in the world. An important question is whether it 
is possible to exploit this data base more fully to examine important questions around 
what works, especially in a number of areas of teaching and learning strategies which 
have been around for a long time but for which there is not much evidence on 
effectiveness. 

First, I would argue that the existing data base could likely be more fully exploited to 
shed light on the differentiation of teaching and learning strategies already referred to. 
With such a large number of projects, it seems likely that fairly exact replications of some 
treatments could be found. Also, even inexact replications can help determine the extent 
to which a treatment is generalizable or is specific to particular subgroups. As Cycle 3 
comes to a conclusion, more projects will be available for analysis and there is time to 
rethink the level of analysis that appears in the provincial reports. 

Second, a case can be made that Cycle 4 offers a more direct opportunity to design at 
least some projects as randomized clinical trials. Despite the possibility of replication, 
the before/after approach of AISI is limited as a means of developing causal inferences. 
Given the availability of large numbers of students and the impossibility of implementing 
a treatment for all students at once, RCT designs should be no more difficult to 
implement in a school district than in a medical setting. We often hear that this cannot be 
done because it amounts to depriving students of the treatments. The answer to this is 
obviously that, unless we already know a particular treatment to be effective, any one 
treatment should be viewed as being as effective as any other. Children are not deprived 
of schooling, any more than cancer patients are denied treatment, as a result of being part 
of a RCT. RCTs should be viewed as trials of equally effective treatments, until it 
becomes clear that one treatment has an advantage. 

A related question is whether Cycle 4 should capitalize on what has been learned from 
earlier cycles. Have some specific interventions been established as sufficiently 
promising to warrant their continuation under more tightly controlled conditions? Are 
some inventions so unpromising that they should not be continued under any conditions? 
Given the large investment in AISI, it would certainly be reasonable for the public to ask 
if there is sufficient payoff from what has been done thus far to warrant continuation. 
Since that question has been settled with the announcement of support for Cycle 4, an 
argument can be made that Cycle 4 should not represent either more of the same or 
entrenchment of projects that have been in existence but have not been shown to be 
effective. Alternatively, if any interventions have been clearly demonstrated as effective, 
these should become part of the regular operation of the school system. AISI should not 
be treated as a vehicle for supporting the regular operations of the school system. 
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Taking this a step further, we heard a great deal in the first day of this colloquium about 
gaps in the Alberta system. Despite its acknowledged high performance by Canadian and 
even world standards, it is somewhat disconcerting to hear that Alberta continues to have 
a low high school graduation rate and significant challenges in meeting the needs of 
Aboriginal students. A case can be made that, since average achievement in Alberta is 
high, the focus should shift from the average to closing gaps in achievement. Focusing 
on the performance of the lowest achieving students might, in fact, serve the dual purpose 
of, to use Doug Willms’ term, “raising the bar and closing the gap.” 



Rethinking the Model 

More generally, I would suggest that something could be learned from an organization 
such as the Children’s Oncology Group about a consortium approach to innovation. One 
of the significant differences between research in education and in other applied fields is 
that educational research has a strong jurisdictional component. Because the research 
community is separate from the practitioner community, gaining access to the system for 
research is complex and generally leads to studies being confined to a single jurisdiction, 
most often a school or school district. Only occasionally, as in Tennessee STAR, is a 
larger jurisdiction involved and almost never do we find studies other than surveys which 
are transjurisdictional. 

It seems to me that we have in AISI the potential to transcend the usual school district 
boundaries in conducting research. AISI funding and infrastructure extends to all school 
districts in the province. From what we have heard, many districts are experiencing 
common problems. Certainly, the range of teaching and learning areas that are being 
investigated in many AISI projects have much in common. Greater standardization of 
treatments across districts and engaging in more systematic RCT-like trials on a larger 
scale is in keeping with the idea of selecting a narrower range of projects for more 
detailed study in the next cycle. 

The separation of research from practice is another area that can be addressed through a 
consortium approach. Without stretching the COG analogy too far, a key feature of that 
model is that there is no distinction between practitioners and researchers. Many of the 
same physicians who treat the patients also serve on the committees designing the studies 
and these physicians are designated as principal investigators within their local settings. 
A parallel, which draws from the ideas of action research, is that key teachers could be 
designated as researchers with these teachers forming the nucleus of research teams 
which would take responsibility for both study design and execution of projects within 
their own school or district. This would facilitate a key element of effective large scale 
clinical trials, that of central design and distributed execution. Teachers would also have 
a stake in ensuring fidelity of implementation and other features of design, as well as in 
dissemination of results. AISI seems to be one of the few programs where sufficient 
resources exist to give teachers time to act as researchers. Using the opportunity to forge 
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a closer link between research and practice would be a major contribution to how we 
think about the value and use of evidence in support of educational practice. Just as 
Alberta has used its resources to create a significant presence in medical research, the 
opportunity exists to make the same mark in education. AISI seems to me to offer the 
beginnings of a structure that could allow this to happen. 
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Chapter 15 

The Present and Possible Theory of Change for AISI 

Andy Hargreaves 
Boston College 



The Alberta Initiative for School Improvement (AISI) represents a remarkable and 
unique approach to educational change. By focusing on school-based improvement, 
according to school authority-decided priorities, in cycles that have come to involve 
almost all of the province’s schools rather than just a few already well-disposed 
enthusiasts, the initiative strives to promote change that benefits student learning and that 
energizes teachers’ commitments and capacities. Each school authority determines 
measurements of progress to provide a sense of focus and urgency so a project maintains 
purpose, direction, and transparency. Networking school authorities together through 
annual AISI conferences and other activities, the AISI partnership increases the capacity 
for schools to learn from each other and also attends to efforts to create coherence across 
the system. 



Theories-of- Action in Educational Change 

AISI possesses a distinct theory of change-in-action. All efforts to bring about personal, 
organizational or social change are predicated on more or less explicit beliefs and 
assumptions about how people change and what induces them to do so. Reality TV 
shows that turn around failing restaurants or ineffective parents involve initial evaluation, 
public humiliation, coaching and practice, more humiliation, and final success as people 
progress through the stages of imposed change. Policies that emphasize failure more than 
success, and that impose stringent performance targets for improvement, follow a similar 
pattern. 

Less spectacular efforts to improve are also based on theories-of-action about how people 
change. Identifying exemplary schools or programs assumes that people can copy the 
best. Advocates of scripted curricula and imposed performance targets assume that 
people cannot be trusted to improve by themselves and need others to monitor and 
micromanage their actions in detail. Highlighting exceptions of schools or districts that 
perform against the odds either assumes that if one can be exceptional, anyone can; or 
that conversations with those who are exceptional can spread modified changes around a 
system. Technology buffs imagine that gadgets alone will alter how people learn and 
teach. Coaches suppose that people trying new practices require support and also 
supervision in order to succeed. Leadership theorists either search for charismatic heroes 



AISI Colloquium 



127 



Large Scale Improvement 



who can bring about miraculous change or invest in wider distributions of leadership that 
are able to sustain change over time. Network-driven change, by comparison assumes 
that change spreads like benign infection, where people catch it from each other. 



Transplanting Change 

One of the greatest challenges of educational change is not just how to make it last, but 
also how to get it to spread beyond a few isolated examples of innovation. But most 
imposed reform design is like ripe fruit. It usually does not travel well. In a classic set of 
studies, Mary K. Stein and her colleagues (2004) examined the destinations and destinies 
of originally successful reforms first designed for New York District 2 in the 1990s. With 
a tight and detailed design focused on specified literacy instruction, learner-centered 
leadership, intensive coaching, and a relentless preoccupation with results, a successful 
reform in New York District 2 was transposed, along with some of its architects and 
implementers, to the city of San Diego. After some initial increase in measured 
attainment, the attempt to impose an instant solution on San Diego that had been 
developed over many years in New York was then declared a failure. The researchers 
identified many reasons for this including: 

• Military-based and larger San Diego was more conservative yet had less local 
capacity than smaller District 2 within high-capacity, chutzpah-like New York. 

• San Diego’s reforms were imposed in two years, whereas New York’s had been 
developed over a decade. 

• Large and complex secondary schools were included in the San Diego reform, unlike 
District 2. 

• As San Diego’s reform mill became increasingly gruelling, resentment grew against 
the interlopers responsible for its implementation. 

• Understandings of literacy and instruction that had taken a decade to develop in District 
2 were interpreted more superficially in the fast-track reform environment of San Diego. 

Stein and colleagues document that a little less was lost in translation with a further attempt 
at implementation in Philadelphia as implementers tried to be more sensitive to differences 
of context. Paradoxically, though, it has only been later when the transpositions have been 
less forceful and based more on general principles than specific designs that the District 2 
change has started to travel and spread more successfully. 

How to make reform spread and last has been a preoccupation of policy makers and 
change agents over half a century. It has given rise to distinctive change designs - each 
containing implicit or explicit assumptions about how to bring change into being. I have 
documented this in research on educational change over more than 30 years in eight 
Canadian and US high schools, four of them innovative and four traditional (Hargreaves 
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& Goodson, 2006; Hargreaves & Fink, 2006). This research has revealed several ways of 
change through which schools and school systems have passed over these years, each of 
them leaving legacies in the present, each of them carrying elements that are worth 
retaining and revisiting, as well as elements that should be left behind. AISI has been 
established within and in the wake of this historical legacy of educational change. 



The Three Ways of Change 

The First Way of change comprised the welfare states that culminated in the post-war 
initiatives of the 1960s and early 1970s that provided unprecedented levels of support and 
opportunity for the poor. Yet such policies became expensive after the onset of the first 
world oil crisis in 1976 and when salary levels of state professionals rose as they gained 
seniority. Even worse, they sometimes helped foster long-term state dependency among 
the intergenerational poor without providing any real foundation for their long-term civic 
engagement. 

This First Way gave state professionals considerable freedom. My own research revealed 
that teachers of the baby boomer generation bemoan the loss of this professional autonomy 
- but for different (and even opposing) reasons. Some are nostalgic for freedom to pursue 
compelling world-changing missions through their practice and to respond flexibly to the 
diverse learning needs of their students. Others are nostalgic only for the loss of their 
subject freedom irrespective of whether it met student needs at all. The First Way in 
education therefore fostered considerable innovation but it also allowed unacceptable 
inconsistency in quality. 




Figure 15.1. The First Way 
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In the Second Way of Reaganism, Thatcherism, and even the early days of Kleinism in 
Alberta, anti-tax governments cut social services and outsourced others to the private sector 
to reduce costs and dependency and encourage entrepreneurial drive and individual 
responsibility. But they also undermined social cohesion and widened the gaps between 
rich and poor. For example, while Canada ranks in the middle of developed nations on 
UNICEF’s indicators of child well-being, there is a divergence between very high scores on 
material well-being and educational well-being as measured by achievement tests on the one 
hand, and family, peer-related, and self-assessed well-being on the other (UNICEF, 2007). 

In The Second Way, professionals came under increasing regulation by the market and the 
state. Educators experienced a collapse of professional motivation, leading to crises of 
teacher retention and leadership renewal, as high- stakes standardization, driven by 
government performance targets, and inter-school competition, pulled the passion and the 
pleasure out of teaching, contributed to union strife, and increased the workload and 
vulnerability of those engaged in leadership (Hargreaves, 2003). 



Top-Down Government 




Support-Bottom UP 

Figure 15.2. The Second Way 



A Third Way advocated by Bill Clinton and Tony Blair along with other world leaders, 
and theorized by Anthony Giddens (1999) offered something between and beyond the 
state’s all-providing First Way and the linking of markets to standards and targets of 
performance in the Second. It proposed a mixed economy of diverse providers in 
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education, leading to the increasing prominence of charter schools in North America and 
specialist schools in the United Kingdom. In places like England, Ontario, and also 
Alberta, support for public services, especially education, increased in terms of financial 
resources, buildings, materials, and training. Professional motivation is being restored 
not by giving back unconditional freedom to teachers, but by strengthening professional 
learning communities, and networks where professionals learn from professionals and 
schools help each other to improve - injecting lateral energy into the system (Fullan, 
2005; Levin, 2008). 

There is more support and more professional energy in the Third Way. These are 
undoubtedly good things. But there are worrying developments, too. Much of what has 
passed for the Third Way has been the emergence of technological and data-driven self- 
surveillance among educators to deliver unchanging government goals. Literacy and 
numeracy remain pre-eminent in provinces like Ontario and British Columbia even when 
systemic achievement in them is already high by international standards. Political targets 
in tested basics still drive the hurried interactions of data-driven professional learning 
communities as teachers anticipate the upcoming test dates in their preparations, 
curriculum focus, training choices, and classroom activities. 

In the Third Way, bottom- up support and lateral networks have some success in securing 
short-term test gains, but the political culture of imposed targets and testing undermines 
longer term or more innovative efforts, that are essential in a competitive and creative 
knowledge economy. The Third Way was meant to be a strategy of development and 
learning. In reality, in many places, it has turned into a system of testing and delivery. 
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Figure 15.3. The Third Way 
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The Alberta Initiative for School Improvement contains elements of all three ways of 
change. Indeed, its evolution through its first three cycles almost seems to replicate aspects 
of the much longer cycles of the three ways of change. The whole conception of local 
innovation is very much a First Way philosophy. Resources are provided to schools to 
support and stimulate local innovation and there is a belief in the power of professional 
commitment along with the existence of or ability to develop teacher capacity as the key 
driving force of improvement. 

Some of the school authority presentations and discussions at the AISI Colloquium gave a 
hint of a first cycle where a thousand flowers could bloom leading to great variety on the 
one hand, while repeating the First Way’s problems of lack of coherence or consistency on 
the other. In this first cycle, colloquium members learned, innovations often focused on 
safer areas that did not address or expose teaching and learning practices, such as 
technology or improving student behavior and school climate. At the same time, principals 
and school district leaders reporting on their work pointed out that these safer starting 
points allowed collaborative habits to be established and professional trust to be built, 
laying a foundation for more challenging efforts at innovation later on. Deep and 
sustainable change takes time, and the architects of AISI have been the protectors and 
beneficiaries of it. 

In the second cycle of AISI, we were informed, some districts tended to overreact to the 
lack of clear focus in Cycle 1, by imposing a focus on all schools in Cycle 2 - matching 
the prescriptive reactions of the Second Way to the diffuseness and inconsistency of the 
first. Here, participants concluded, the pendulum sometimes swung too far in the other 
direction, restricting the very creativity and diversity that AISI was meant to foster in the 
first instance. 

In the third cycle, AISI has contained many elements of the Third Way and, in some 
respects, exemplifies Third Way change strategy at its best - though it also suffers from 
some key frustrations of the Third Way. AISI exemplifies Third Way thinking in its very 
establishment of a partnership between Alberta Education and the AISI education 
partners. The Government and the AISI partners collaborate to bring about positive 
change that engages and is driven by all partners. Additional, though not copious 
resources are invested in educational improvement through AISI. In education as in 
business, achievement requires investment. In AISI, as in the provincial strategy 
generally, there is strong support for teacher collaboration in professional learning 
communities of improvement and inquiry in ways that improve learning and lift 
achievement. The focus on learning and achievement is more explicit than in AISI Cycle 
1 and First Way activity. Evidence and results matter in AISI and schools are required to 
document the effects and impact of their innovative efforts. This introduces an element 
of accountability that surpasses Cycle 1/First Way thinking, but that also avoids the 
Second Way’s and second cycle’s reliance on standardized test and examination results 
alone as measures of impact and improvement. 
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The AISI Colloquium along with supporting documentation also highlighted some of the 
limitations of AISI’s strategy and development as well as some inherent dilemmas it must 
confront as it moves forward into Cycle 4. 

• There is a tendency for professional collaboration to be too courteous and polite and 
to avoid issues of controversy and challenge that create professional discomfort. As 
the poet Virgil pointed out, even our enemies have things to teach us. Similarly, 

Doris Kearns Goodwin (2006), in Team of Rivals, demonstrates how Abraham 
Lincoln led by pulling together people of competing dispositions and ideologies. 
Cycle 4 might be a place for this more robust professional dialogue to occur. 

• Collaborative improvement efforts rarely extend beyond professionals to encompass 
other adults in the community. The school still has the smallest effect size on student 
achievement. Poverty, infant health problems, limitations in parental education, and 
rival isolation still have the greatest collective restricting influence on learning and 
achievement. Only by principals taking on roles as community leaders in ways that 
build parent and community involvement and engagement, can this statistic be 
transformed, so that schools affect the things that affect them. 

• AISI has successfully combated teacher isolation on a significant scale but it has not 
yet combated school isolation. Evidence of interschool visitation and assistance is 
not strong. Annual or even twice-yearly conferences are not sufficient to circumvent 
this problem. It may be that districts will provide these supports, but some districts 
are small, and depending on the quality of system leadership, not all districts will be 
effective in developing system-wide collaboration. Moreover, districts may be 
tempted to turn local school networks into local clusters to implement district or 
Ministry policy - damaging the very spirit of innovation and independence that AISI 
represents. 

• AISI operates within a continuing and constraining context of high-stakes testing in a 
limited number of curriculum areas and this creates tension about how AISI’s impact 
should be measured. Government and often district officials prefer to document the 
impact on standardized, province-wide achievement tests, while teachers, their 
representatives, and others argue that not all AISI innovations directly address 
achievement in the areas that are measured, and that more qualitative measures 
should be considered as well. 

As AISI moves into Cycle 4 and faces these challenges, it might want to consider also 
embracing a Fourth Way, rather than the Third Way of educational change (Hargreaves 
& Shirley 2008; 2009). Some clues as to what this may might look like can be gained by 
examining three alternative paths of improvement from overseas. 
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Three Alternatives 



Elements of Fourth Way thinking are not just hypothetical or theoretical, but already exist 
somewhere within our own systems or in places beyond them. Three are presented here - 
a top-performing country, a dramatically turned-around district, and an innovative and 
successful network. 



Finland - A High Performing Country 

Finland rank s No. 1 on most PISA assessments of international achievement, has the 
narrowest achievement gaps in the developed world, and is a world leader in corporate 
transparency and economic competitiveness. It is one of the very jurisdictions that out- 
performs Alberta. In 2007, 1 took a team to Finland for the Organization for Economic 
Cooperation and Development to examine the relationship between leadership and school 
improvement (Hargreaves, Halasz, & Pont, 2008). Drawing on our evidence and on the 
growing body of other literature on the Finnish experience (Sahlberg, 2006; Aho, 

Pitkanen, & Sahlberg, 2006; Castells & Himanen, 2004; Grubb, 2007), this is what we 
concluded: 

• After being one of the most backward economies in Europe in the 1950s and after an 
international banking crisis, the loss of its Russian market, and the escalation of 
unemployment rates to almost 19% in the early 1990s, Finland consciously connected 
economic transformation towards being a creative and flexible knowledge economy 
to the development of a significantly more decentralized education system. 

• This effort has been coordinated at the highest political level where CEOs from 
leading companies like Nokia meet regularly with university presidents on a science 
and technological development committee chaired by the Prime Minister. 

• The coherence is not merely bureaucratic and governmental, but visionary and 
inspirational. Finns have a common vision that connects their creative high-tech 
future to their past as a creative, craft-like people. There are more orchestral 
conductors per capita in Finland than any other developed country, and all young 
people engage in creative and performing arts to the end of their secondary education. 

• This vision is shared at every level among Finns since teachers create their country’s 
future as a creative and inclusive nation. Though paid only at the OECD average, 
teaching in Finland is highly competitive with only a one-in-ten chance of acceptance. 
Retention is high among Finnish teachers because conditions are good and trust is high. 
All Finnish teachers are awarded Masters degrees. Finns control quality at the most 
important point - the point of entry. 
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• Within broad guidelines, or steering by the state, highly qualified teachers create 
curriculum together in each municipality for the children they know best. Curriculum 
and pedagogy are not separated - they are in a common tradition of what continental 
Europeans call “didactics”. The sense of delivering a curriculum devised by others 
from afar is utterly alien to Finnish educators. 

• In small classes rarely larger than 24, and with generous definitions of special 
educational need, the push for quality is driven largely by quietly lifting all children 
up from the bottom, one at time, through knowing them well in small classes, having 
specialist support as needed, and not having to deal with excessive paperwork and 
endless external initiatives. 

• Principals work across schools, sharing resources where they are needed, and feeling 
responsible together for all the children and young people in their town and city, not 
acting competitively only for the children in their own school. 

• Assessment strategies are largely diagnostic forms of assessment for learning and 
internal to the school. External accountability is confidential and undertaken on a 
sample basis for monitoring purposes only, not as a census of everyone. 

• Principals are seen as being part of a “society of equals” in their schools, not as line 
managers. They are often recruited from within their schools, and they engage in 
considerable informal distributed leadership with their colleagues. It is illegal for a 
principal to be recruited from outside education, and all principals teach for at least 
two hours per week. Teachers say that if the principal is indisposed or ineffective, 
they take over the school as it belongs to all them. 

What might intelligent engagement with Finnish strategies look like? The most 
transferable aspects are the broad principles of developing an inspiring and inclusive 
mission that attracts high calibre people into the profession capable of creating 
curriculum together for children they know well in smaller classes. 

Without an inspiring and inclusive mission, other less successful measures such as 
market incentives have to be used to attract and retain highly qualified professionals. 
Without highly qualified professionals, teachers cannot be trusted so much, which 
increases the argument for external accountability, standardized curriculum, and 
government intervention. But these measures then destroy nations’ capacities to be 
competitive and creative knowledge economies. Last, without small classes in which 
teachers know their children well, individual knowledge of children’s needs has to be 
developed in other ways, through batteries of data on standardized tests. 
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Tower Hamlets - A Turned-Around District 



London’s Docklands are squeezed into a tiny peninsula that occupies one of the last 
snaking bends of the River Thames before it opens out into the English Channel. After 
the collapse of the dockworkers industry when large oil tankers could no longer navigate 
the River Thames, the Docklands became a residential area occupied by poor, largely 
Bangladeshi immigrants, awkwardly juxtaposed right next to the global financial centre 
of Canary Wharf. For more than a decade, this has been the borough and municipality of 
Tower Hamlets. 

Tower Hamlets’ Bengali community suffered from high unemployment rates and some of 
the greatest incidences of poverty in the country with more children on free school meals 
than almost anywhere else. Educators’ aspirations for student achievement were startlingly 
low and in 1997, the Borough was proclaimed the country’s worst-performing Local 
Education Authority, with the lowest performing primary school in the nation. 

Ten years later, the transformation of the schools in Tower Hamlets is dramatic. The 
schools perform around and above the national average. On standardized achievement 
tests, GCSE examination results, and rates of students going on to university, the borough 
ranks as the most improved local authority in Britain. It has significantly reduced 
achievement gaps in relation to children with special educational needs, those from 
cultural minorities, and those on free school meals. In a large-scale research project co- 
directed with Alma Harris called Performing Beyond Expectations, I have been studying 
the secrets of Tower Hamlet’s success 49 . 

While England’s Third Way has endeavored to deliver more children’s services to 
disadvantaged and other communities, Tower Hamlets has worked tirelessly to create 
new capacity to strengthen community relations and engagement. It has done this by 
working with faith-based organizations and forming agreements with imams from this 
largely Muslim community to counter the achievement effects of children taking 
extended absence from schools to attend and then stay on after family events such as 
funerals in Bangladesh. This includes announcements at school and at prayer in the 
mosque that extended absences will be treated as truancy because the educational 
achievement of the young people, and the development of the community’s future 
capacity, matter that much. 

Likewise, Tower Hamlets has developed some of its schools into community centers - 
establishing extended services that keep a school open from 8:00 am until 10:00 pm in the 
evening and providing resources and recreation for both students and the community’s 
adults in one integrated site. 



49 This study is ongoing at the time of writing. It is funded by the UK Specialist Schools and Academies 
Trust, and the National College for School Leadership. 
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One important factor that has strengthened community engagement in Tower Hamlets has 
been the impact of legislation known as Workforce Remodeling. This was designed to place 
more classroom assistants and other staff in schools to support teachers by undertaking 
administrative and other tasks for them. Workforce remodeling has come to mean that 
around half the adults in most schools in England now come from the community itself. 

They are not “qualified” teachers based on the criteria established by the Government, but 
they develop strong relationships with the teachers and leaders of the school and sometimes 
go on to become professionally trained teachers themselves. “We’ve got quite strong training 
and development opportunities for local people,” one Tower Hamlets leader commented, 
who “particularly come through as teaching assistants in our schools to train up to become 
teachers so I think we’ve got a good way of delivering more home grown teachers and social 
workers.” These groups of professionals and employees from the local community working 
alongside each other for the students they share in common build communities of trust, 
engagement, and advocacy that bring about improvement together. One leader stated that it 
was about “creating those links and making sure everybody feels part of one community and 
they have access.” 

So Tower Hamlets uses community development strategies to recognize and develop 
indigenous community leadership, strengthen trusting relationships between professionals 
and community members, and transform schools into vibrant examples of democracy. 

These processes of community development operate in synergy with other transformational 
elements of the district’s improvement agenda. 

• The visionary leadership of a new Director (superintendent) who was a self- 
confessed workaholic and who believed that “poverty is not an excuse for poor 
outcomes,” that aspirations should be extremely high, that efforts to meet these 
aspirations should be relentless, and that everyone should work on this together. 

• The successful succession of this first driving leader by a more developmentally- 
inclined, yet equally persistent one, with just a short period of instability in between. 

• The ability to attract high quality teachers who stay with the borough, after a period 
of weeding out overseas teachers who were drawn more to enjoying a brief life 
excursion in London than a long-term professional commitment to the schools. 

• A commitment developed with the schools’ leaders to set and reach ambitious shared 
targets for improvement in “a culture of target setting” so that “everybody owns 
them”. 

• Intelligent and circumspect use of data - It’s “not just about the data. It’s actually 
knowing the school, knowing the community, knowing about history, knowing about 
the staff - all of that.” 

• A shared philosophy that it is better to have ambitious targets and just miss them than 
have more modest targets and meet them. 
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• Mutual trust and strong respect where “lots of our schools work very closely together 
and with the local authority” and where inspectors’ reports refer to the “enthusiasm 
and high level of morale among the workforce”. 

• Knowledge of and presence in the schools which provides support, builds trust, and 
grounds intervention in consistent and direct personal knowledge and communication 
more than in the numerical data that eventually appear on spreadsheets. 

• A commitment to cross-school collaboration, so that when one secondary school 
went into “Special Measures” (similar to “corrective action” in the US) after taking in 
Somali students from refugee families in a neighboring authority, all the other 
secondary schools rallied round to help. 

• A resilient but not reckless approach to external government pressure and policy - 
accepting the importance of testing and targets, but deciding to set their own targets 
and resisting the politically motivated pressure to build new high school academies 
since the authority already had high-trust relationships with its schools that now 
performed very well. 

• The economic regeneration of sections of Canary Wharf that has created an 
environment of hope and well-being through positive business partnerships that 
model a new form of “corporate educational responsibility” with schools. 

An Innovative Network 

With Dennis Shirley, I recently evaluated the performance of more than 300 secondary 
schools in the Raising Achievement / Transforming Learning (RATL) project of the 
English Secondary Schools and Academies Trust (SSAT) (Hargreaves, Shirley, Evans, 
Johnson, & Riseman, 2007). Two third of the schools improved at double the rate of the 
national average over two years through having a specific network architecture which 
involved: 

• Schools helping schools in peer-driven networks of lateral pressure and support 

• Combining outside-in knowledge of experts at conferences, with inside-out 
knowledge of successful and experienced practitioners 

• Offering mentor schools to lower performing peers (rather than mandating them) in 
cultures of strong expectation for improvement within transparent lateral systems of 
involvement 

• Supplying modest amounts of additional resourcing to facilitate these improvements 
and interactions 



AISI Colloquium 



138 



Large Scale Improvement 




• Providing clear, practical menus of short-term, medium-term, and long-term 
strategies for improvement and transformation with proven success among 
experienced administrators 

In terms of general principles, this policy initiative points to the success and promise of a 
professional, peer-driven strategy of the strong helping the weak in cultures of committed 
and transparent improvement. It points to the power of lateral professional peer interaction 
and collaboration as a way to bring about large scale improvement when this involves 
transparent processes of engagement as well as mutual visibility of progress and results. 



Fourth Way Thinking 

These three alternatives point to a set of distinctive directions in Fourth Way thinking and 
change strategy that overlap in some places with Third Way strategy, but depart from it 
significantly in others. This Fourth Way of change brings together an energized profession 
with an engaged public and a guiding but not controlling government, in an interactive 
partnership of equals dedicated to serving and improving the public and educational 
common good. These principles include the importance of: 

• A compelling, inclusive, and inspirational vision in the society and its schools 

• Learning and achievement priorities that follow the vision 

• Attraction and retention of high quality teachers 

• Professional cultures of trust, cooperation, and responsibility 

• Ambitious and shared targets rather than arbitrary and imposed ones 

• Evidence-informed rather than data-driven improvement 

• Sophisticated rather than singular and standardized systems of accountability 

• Close relationships of mutual trust between districts and schools 

• Professional networking of peers and with mentors across and beyond districts 

• Cultures of improvement where the strong help the weak 

• Community development, engagement, and empowerment 

The contrast between the Third and Fourth Ways is represented in Figure 15.4. 
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Figure 15.4. Third Way to Fourth Way Solutions 
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Implications for AISI 

What might the Fourth Way offer for AISI as it moves into Cycle 4? 

First, AISI must celebrate its considerable achievements so far in the improvement results 
that are documented in the accompanying chapters and it must recognize the significant 
inroads it has already made into Fourth Way strategy - being one of the world’s leading 
exemplars of educational change that energizes the profession and improves student 
learning on a large scale. The partnership between the Alberta Government and its AISI 
partners is unparalleled internationally in the way it engages all partners as agents of 
positive change. This demands much of all partners. For example, the Alberta Teachers’ 
Association, unusually among its international counterparts, is not a traditional opponent 
of change, nor even an organization that has become complicit with change in exchange 
for other negotiated benefits, but it is a vigorous agent of and active partner in bringing 
about demanding changes that benefit all students. 

AISI school authorities and schools treat data seriously but in ways that make sense for 
the individual schools and there is evidence of a target-setting culture emerging where 
schools and teachers are transparently responsible for their own progress over time. Its 
strategy of bottom-up innovation recaptures the best spirit of the First Way, but ensures 
more widespread impact through the AISI partnership and coordination via the AISI 
leadership team. Districts and schools also have occasional opportunities to come 
together annually to share strategies and successes. 

There are six areas where AISI can and perhaps should push further as it moves into 
Cycle 4 of funding, in ways that would be consistent with the Fourth Way approach to 
educational change that was exemplified in the three dramatically contrasting cases of 
innovation and success discussed earlier. 

First, AISI has developed increasingly powerful forms of collaboration within schools. It 
should now develop even stronger networks to connect school authorities and schools 
with each other across the province to strengthen professional support, increase learning 
and circulation of effective strategies across the system, and provide ways for schools and 
professionals to support, pressure, and challenge each other to improve. These networks 
should, if they are to conform to effective network theory and practice, have: 

• A clearly defined architecture 

• A strongly agreed common purpose within the network as whole or within subsets of it 

• Agreed norms of participation 

• Transparency of participation and outcomes 

• Means for mutual assistance 

• Useful tools and artifacts to share such as lesson plans or strategies for involving parents 

• A world class website and web portal for schools to communicate with each other 
independently 

Networks are digital systems of communication and they will and should challenge the 
analog systems of government that are typically used to manage and implement policy 
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through chain-like systems of command running through school districts. In this sense, 
networks will be a disruptive innovation for the province, establishing autonomous lines 
of communication and exchange among schools, independent of direct Government or 
district control. This will require a discourse of professional empowerment and a funding 
model that allocates resources not just to the school authorities and therefore also schools 
for within- school development, but also to the costs of schools connecting with each 
other across authorities. 

Second, AISI will fare better if it is located within a provincial educational and political 
vision that is inspiring and inclusive in the context of a post-oil economy. This vision will 
need to embrace the 21 st century skills essential to developing an advanced knowledge 
economy, and to address the need for greater social cohesion and community development 
in a province and country that scores poorly on many key measures of child well-being. 

This will more readily align AISI with a broad range of challenging educational outcomes 
and ease some of the feelings of tension with a provincial accountability agenda that, 
many feel, is more directly concerned with measured achievement results in a more 
limited number of areas. 

Third, AISI projects do not always directly address curriculum areas such as literacy 
which can be directly assessed in relation to provincial achievement results. It is 
important in these instances not only to devise other assessments of impact appropriate to 
the projects concerned, but also to develop shared proximal indicators or milestones of 
progress towards more complex, longer-term outcomes, as ways of building confidence 
and capacity among staff, and of establishing accountability among parents and the 
community. Before gains in measured achievement results, for instance, one might first 
see changes in awareness and beliefs and then practices among teachers, followed by 
changes in levels of engagement among students. 

Fourth, some schools perform better than others working with similar kinds of populations. 
Part of the challenge of systemic improvement and of achieving social justice in student 
outcomes and opportunities, is to get over the polite pretence that all teachers are equally as 
good, or basically the same. It is important to have ways to identify differences between 
schools or programs, so that stronger partners can assist struggling peers. The networking 
together of AISI schools is therefore a challenge not only of connecting schools diffusely, 
but also of establishing specific connections between high-performing schools and those 
that are doing less well. At the moment, these processes, if they occur at all, occur within 
districts. But the best sources of help, or availability of successful strategies may and often 
do exist across district boundaries. One Fourth Way direction for AISI therefore is to 
establish and make professionally available assessment information that differentiates levels 
of performance between schools on key criteria not as a form of competitive ranking, still 
less of public shaming, but as a clear device to channel help through the network to where it 
is needed most. 

Fifth, while all AISI projects formally require some degree of parental involvement, few 
currently place their main focus on improving relations with parents and communities. 

Yet these are absolutely central to student learning and achievement. In Finland and 
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Tower Hamlets, parent and community engagement was absolutely integral to 
improvement and effectiveness. In both cases, this amounted to more than redoubled 
efforts to communicate more often, improve voicemail systems, initiate home visits, or 
design better parents’ evenings. Finland has a culture of high commitment among parents 
towards taking responsibility for their own children’s development long before school 
begins. Tower Hamlets transformed the dynamics of parent-teacher trust and community 
involvement by creating a new structure for parent and community employment in the 
schools. Increased teacher autonomy from government must be matched by decreased 
autonomy from parents and communities. Bureaucracy can only be ceded to democracy. 
A challenge for AISI in Cycle 4 is how to increase parent and community engagement 
along with a more open professionalism among teachers who are able and willing to work 
effectively alongside parent and community members. 

Sixth and last: AISI is now so well developed and has been sustained for such a long 
period that the time must be approaching when it is regarded not as a supplement to 
government policy but as a set of practices, of ways to manage change, that become 
integral to the very core of how Government operates and makes change itself in 
education. At what point should the Alberta Initiative for School Improvement become 
the Alberta Strategy for School Improvement? 



Conclusion 

As a critical friend of the Alberta Initiative for School Improvement, my task has been to 
identify and comment on its theory and practice of change and to compare it with change 
theories and practices elsewhere in time and space. I have examined AISI in relation to 
the past and present three ways of change that have dominated educational policy since 
World War II. I have set it alongside instances of change elsewhere in exemplary 
countries, districts, and networks. In particular, I have reviewed and commented on 
possible future directions for AISI as it enters Cycle 4, in relation to an emerging Fourth 
Way of educational change appropriate for high-performing schools and school systems 
in a sustainable knowledge society. 

AISI, it has been found, has an internationally unique and remarkable theory of action that 
encourages and achieves school district and school-based bottom-up innovation across all 
school authorities in the province. It has the innovative spirit of the First Way without the 
inconsistency. It is founded on a genuine partnership among AISI partners, not an uneasy 
alliance or entrenched state of conflict. From the Third Way, it exemplifies the power of 
professional learning communities to bring about collaborative change within schools and 
the importance of using a range of self-determined and self-selected measurements and 
indicators to prompt further inquiry and document progress. 

Cycle 4 of AISI is a fitting time to consider how it moves beyond its focused concentration 
on within-district improvements. More robust and challenging dialogue, clearly structured 
networks, broader forms of accountability coupled with proximal measures of progress, and 
significantly greater community engagement - all within a provincial context that 
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establishes an inspiring and inclusive vision for social and educational development in a 
post-oil economy - these are the developments that might ultimately justify AISI moving 
from being a change initiative that supplements Government policy, to a change strategy 
that is an integral part of that policy. 

The economic success of Alberta’s past and present is and has been grounded in its natural 
resources. Alberta’s future, however, will depend on its human resources and on the pools 
of talent, innovation, and creativity that can be released from them. The current change 
strategy of AISI and its development into a more advanced, networked, and integrated part 
of the policy culture of Alberta can be a vital and vigorous part of that future. 
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Chapter 16 

I Have a Dream: School Improvement for Enriching Communities 

Pasi Sahlberg 

European Training Foundation 



Introduction 

Education policies that steer how school improvement is arranged often only marginally 
address the overall aim of developing education systems. Formal goals are expressed 
using terms such as “raised student achievement” or “closing the achievement gap” by a 
given time and at specified levels of schooling. School improvement policies seldom get 
into describing what the key conditions of reaching the set goals are. This is thought to 
be the task of schools and district education authorities. 

I argue that we need to have a dream for school improvement together with official policies 
and set targets. I also challenge conventional conceptions that technical objectives and 
goal statements for school improvement are sufficient to keep the momentum of change 
and to serve as a source of positive energy that is a necessity in any renewal, whether 
institutional or personal. Based on these arguments, my view of school improvement 
policies should aim at building enriching interaction, or more concretely enriching 
communities, in our schools in order to promote risk taking, creativity, and innovation in 
our schools - and ultimately the pursuit of happiness and well-being. 

First, I present a brief retrospective reflection on the evolution of school improvement 
policies in Finland since 1988. Finland is an attractive example of an education system that 
has been able to create high-performing schools and widespread equity at reasonable cost 
in a relatively short period of time. The three phases of school improvement presented here 
hopefully serve as landmarks to think about the history and future of the Alberta Initiative 
for School Improvement (AISI) as well. Then, I describe what an enriching community is 
as a goal of school improvement, and how it might be created in schools in general. Third, 
drawing on my experience and knowledge of AISI, I propose four policy principles to 
enhance school improvement hoping that this resonates with the current thinking in 
Alberta. Finally, I explain my dream of school improvement and how it might come true. 
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Evolution of Educational Change and School Improvement in Finland since 1988 

Understanding the recent history of educational change in Finland is essential in 
explaining why Finland has been able to climb to the top of schooling. Let’s take a brief 
look at Finland’s school improvement policies and related implementation strategies to 
help us in thinking about future options in other places. I have chosen 1988 as a starting 
point for this analysis because it was a particularly good year for Finns: the economy 
was booming, the political system was stable, the welfare state project was completed, 
and the national hockey team won its first medal ever at the Winter Olympics in Calgary. 
In brief, Finland was living a year of success and fame in many other areas expect in 
education. In 1988 Finnish education was rated as average internationally. In 
mathematics and science, Finnish students were performing below international averages 
(Sahlberg, 2007). 



Phase 1: Building an Epistemological Foundation 

The first phase, from 1988-1994, was a time of building an epistemological foundation 
for school improvement The main focus in school improvement was on answering the 
questions: What is knowledge? What is learning? By 1988 the new comprehensive 
schools had reached the age of adolescence and it was time to look critically at how they 
had succeeded in creating knowing and learning. School practices were strongly 
criticized by researchers and also by some experimental practitioners as being outdated 
and focused too much on direct teaching and as a consequence rote learning. 

In 1988 and 1989 three significant and easy-to-read publications for teachers were 
published. The first one, The Existence of the Dominant Conception of Knowledge in 
School Practices 50 , was an empirical study of how curriculum, textbooks, and school 
organization all reflect static and transmission-oriented conceptions of knowledge at the 
dawn of a knowledge society and breakthrough in complexity thinking - both based on 
dynamic and nonlinear ideas of knowledge. The second publication, The Conception of 
Learning, described the contemporary theories of learning from the perspective of a 
dynamic conception of knowledge presented in the first book. This was a strong message 
to teachers to draw a line between teaching and learning, and to question any general 
direct causal relation between the two. The third publication, The Conception of 
Knowledge, followed the previous dialogue and built a bridge and also an operational link 
between the epistemological features of knowledge and learning. 

These three books quickly became bedtime readers for many teachers. Staff development 
programs were filled with presentations and hands-on activities to translate the new 
interpretations of knowledge and learning into school practice. All this happened on the 
eve of what became the most fundamental curriculum reform in the history of Finland in 
1994. Until then, schools followed centrally mandated and prescribed curricula that left 



50 These titles are all translations of Finnish publications. 
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only a little room for any local or individual solutions regarding teaching. Now, with the 
new National Framework for Curriculum that was built upon the epistemological foundation 
(i.e., dynamic conception of knowledge and constructivist approach to learning), teachers 
and school principals took a leadership in curriculum design. Partly because of this, the 
1994 curriculum reform was adopted with enthusiasm and interest by most schools; about 
four out of five teachers in comprehensive and high schools took part in their own school or 
municipality curriculum development and related professional upgrading of teachers 
knowledge and skills. 

An important aspect of curriculum development was to interpret the epistemological 
foundation rebuilt during the past few years in a way that helped teachers plan their 
classroom teaching accordingly. School curricula, that were approved only by the 
municipal education authorities, often explicitly described intended teaching and in 
many cases also school organization as creative and constructive process or place. 
Epistemological revisions of school pedagogy during this phase led to an active search 
for classroom practices and new school organizations that would support 
implementation of the emerging conceptions of knowledge and learning. This phase of 
school improvement created a bedrock that has remained an intellectual foundation for 
teachers and schools. 



Phase II: Open School Improvement Networks 

The second phase, from 1994 to 2001, was a time of open school improvement networks. 
The new curriculum that was gradually rolled out to all schools from August 1994 offered 
an opportunity to make changes in school improvement policies as well. Until then, 
teachers had mainly followed inservice training that was brought to them by the authorities, 
often as obligatory training days in the municipalities. A revolutionary thought at that time 
was to establish an open network of teachers, principals, and schools that would serve as a 
platform for exchanging ideas, experiences, and tools to respond to emerging challenges in 
the new curricula, especially to those originating from the renewed conceptions of 
knowledge and learning. Schools greeted this school improvement innovation with delight 
and joined this new network en masse. There were very few administrative procedures in 
becoming a partner in the network. The key conditions were willingness to change, share 
ideas, and encourage others to do the same. The national education authorities who were in 
charge of financing and coordinating this network deliberately lowered the threshold for 
joining in. The main principle was that anyone who accepts the common conditions could 
join - and also leave the network if the school didn’t find it helpful. 

School improvement based on voluntary participation also attracted the Third Sector 
community, that is, leaders of sports clubs, scouts, voluntary organizations, and other 
youth activity groups. A common theme for schools and nongovernmental organizations 
was to offer children the best possible opportunities to learn regardless of where they 
spent their time. The idea that students learn different things in different places and, even 
more importantly, that they can learn similar things in and out of school, brought along 
the idea of school as an activity centre as part of the open school network improvement. 
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The essence of this idea was that school can operate as a centre for several purposes, for 
instance, formal schooling, sports, library or cultural centre. Tens, perhaps hundreds of 
schools adopted this idea and based their school improvement on expanding opportunities 
to learn through partnerships with their Third Sector colleagues. As with many good 
educational ideas, this innovation gradually dried up due to lack of political and 
administrative support from those who viewed schooling as a more formal institution. 

The government finally disconnected itself from this network in 2000. Many activities 
still remain but without coordination, they are now more informal cultures in schools and 
in some municipalities. 



Phase III: Maintaining Success 

The third phase is one of maintaining success that began in 2001. At midnight of 
December 6, 2001 (which happens to be the Finnish Independence Day) the world 
changed. The Organization for Economic Cooperation and Development (OECD, 2001) 
published the results of the first PISA 51 survey, with unexpected results. Finland, viewed 
until then been as an average education nation was the best performing nation in more 
than one category: student achievement, equity, and economic efficiency. This night 
terminated practically all criticism and complaints about the inadequate quality of the 
Finnish comprehensive school. Scores of foreign delegations visited Finland looking for 
the secret to good educational performance. International fame quickly replaced modesty 
and low self-esteem from which Finns are often said to be suffering. Global glory has also 
changed education policy makers - only a few are brave enough to suggest any changes to 
current education. 

In the first decade of the third millennium Finland has established a global reputation as a 
model education nation. There is, indeed, evidence that Finland’s educational performance 
has progressed steadily in terms of international comparisons since the early 1980s 
(Sahlberg, 2007). Mobile phone makers, ice hockey players, symphony orchestra 
conductors, and Formula 1 drivers are marks of what the Finnish culture and society values 
- ingenuity, creativity, and risk taking. The question is, “Will the Finnish education system 
continue to be a model in the future?” 

On the one hand, Finland’s systemic educational leadership since the 1970s, its stable 
political structure, and its established complementarity among public-policy sectors would 
suggest that its educational performance will remain strong 52 . On the other hand, OECD 
PISA results and other education indicators have created a feeling of complacency among 
education policy makers, politicians, and the public-at-large regarding the status of 



51 Editor’s Note. The Programme for International Student Assessment (PISA) assesses reading, mathematics 
and science every three years. Go to http://www.pisa.gc.ca/what pisa.shtml for Canadian results for 2000, 
2003, and 2006. 

52 More about education in Finland and its history can be viewed at www.pasisahlberg.com . 
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Finnish education. This may lead to a condition favoring the status quo, where education 
policies and leadership of a high-performing system are motivated by a desire to maintain 
the current situation, rather than seeing what possible futures might emerge from a 
reformed Finnish education system. 

Educational change in Finland since the 1970s has been driven by culture and emotion in 
the context of social, political, and economic survival (Sahlberg, 2009). Such hype was 
experienced in the 1990s after the Great Economic Recession during Phase II of school 
improvement described earlier. The lessons from Finland are that technical knowledge or 
political interests are not enough to renew society without emotional engagement. This 
may be worth considering when the future of AISI is discussed. Indeed, global education 
reforms show that too rational an approach to change does not work because renewal 
requires energy, and energy is driven by emotion. In an era of Big Changes, emotional 
passion often emerges from crisis - or a sense of survival - as it did in Finland. But it 
can also come from viewing new economic, technological, or cultural opportunities. 

At the beginning of the 21 st century, Finland has become a model nation also for other 
reasons: it has been able to build a competitive knowledge economy while maintaining 
the social justice of the Nordic welfare state model. A high-level think tank named the 
New Club of Paris that considered possible futures for Finland stated that survival is not 
the impetus for renewal anymore to keep all the good that Finland has been able to build. 
In their recommendations to the Finnish Government they suggested that: 

... other drivers with emotional effect need to be identified. The question is how to 
broaden the scale of emotional recognition and exploitation. Instead of survival the 
driver for change could be a powerful vision, or the Big Dream of Finland. If people 
do not love the idea, it is futile to publish new strategies. The new strategy with 
cultural and emotional dimensions should be simple; a couple of words that people can 
immediately and emotionally relate to. This is currently missing. (Stahle, 2007, p. 2) 

The spirit of that general recommendation should also be considered in education, both in 
Finland and in Alberta. The chief instrument that guides Finnish education policies and 
educational renewal is the Ministry of Education’s Development Plan for Education and 
Research for 2007-12. This, as its former document for 2003-08, continues earlier 
policies and principles. These documents emphasize equal opportunities, quality 
education, skilled workers, and developing tertiary education and teachers as main 
resources of education. Furthermore, these documents place strong emphasis on the 
complementarity principle and developing the education system as a whole. All this 
assumes that the Finnish education system will continue to perform well in the coming 
years. However, some trends within the governance of the education system provide 
cause for concern. 

First, national education authorities have tightened control over schools and signaled that 
there is not a high level of confidence in schools’ ability to judge what is best for students 
and parents. For example, the new National Curriculum Framework of 2004 reduces 
schools’ role in curriculum planning. Second, the Ministry of Education’s Education 
Sector Productivity Program for 2006-10 calls for schools to do more with less and 
proposes school mergers (i.e., make schools bigger and optimize class sizes, i.e., make 



AISI Colloquium 



149 



Large Scale Improvement 




classes larger). In some cases, productivity gains are sought by reducing schools’ special 
education and counseling services, after-school activities, and assistant teachers in classes 
where additional help is needed. This may turn out to be harmful for the still rather high 
social capital of Finnish schools. Finally, there is no clear idea within the Finnish 
education system of what the direction of education should be in the future. In other words, 
the compelling social vision of the 1970s has diminished and the inspirational energy of 
1990s is turning into coping with the inevitable. For example, the Development Plan for 
Education and Research for 2007-12 is silent about how education should react to needs 
expressed in the economic sector to intensify innovation and create new products. 

Increasing productivity and improved efficiency lead to financial savings and perhaps 
temporarily better services, but in the economic context the development plan’s strategy 
of shrinking budgets will never create sustainable improvements unless there are 
simultaneous investments in something new. There are enough signals through forecasts 
of the Finnish economy and society to suggest that more investments are needed to create 
new ideas and innovations both in education and in economic development, and to 
maintain the high level of social capital that has traditionally been the driver of strong 
educational performance 53 . Finland was able to benefit from one of the most competitive 
national economies when competition within its education system was minimized in the 
1990s. A component of educational change that creates new ideas and innovation should 
provide enough encouragement and support for risk taking that will enable creativity to 
flourish in classrooms and schools. This is possible only with continuous renewal of 
Finnish education, guided by wise educational leadership in close relationship with other 
public-sector policies. The political and strategic challenge that is both ironic and 
paradoxical remains, “Which measures need to be taken to wake up the Finns for the 
change?” 



Enriching Community and School Improvement 

School is a community of students, educators, and other staff. The culture of this 
community determines, and is determined by values, interactions, habits, and expectations 
of its members. Communication, conversation, and dialogue are the building blocks of a 
culture that builds understanding and sense of belonging to the community, and also 
constructs shared meanings out of individual histories and common experiences. 

Think about good teachers you had in school. Why do you remember them? What made 
these teachers extraordinary? Many of us respond by saying that memorable teachers are 
those who genuinely tried to help me go further than I could have gone alone, or become 
who I am now. This includes, among other things, giving a value to an individual, or 



53 See for example the World Economic Forum’s Global Competitiveness Index 2008-09 
( http ://w w w . weforum. org/en/index . htm ) ; Transparency International’s Corruption Perception Index 2008 
( http://www.transparencv.Org/ l; or National Youth Barometer 2008 
( http://www.nuorisotutkimusseura.fi/sivu.php7artikkeli id=72 ). 
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showing that she or he is important, unique and able to learn. Teachers who are able to 
encourage their students to go beyond their limits are those most often remembered by 
students. These teachers, either consciously or unconsciously, promote enriching 
interaction with their students. Enriching interaction means that a person’s moral 
purpose is to help other people to be more than they could ever be alone, without putting 
her or his own personal interests first. 

The ideal culture of learning is characterized by enriching interaction in which the 
teacher helps the learner find “the fire that is inflamed by a spark that then feed itself’. 
This was Socrates’ basic educational principle as described in Plato’s works. Enriching 
community is a context in which its members are in enriching interaction among 
themselves. In an educational enriching community people, not only teachers, help 
others to go further than where they are or where they could go alone. In an ideal 
enriching community people also contend or strive for who will be able to go furthest or 
find the most creative ideas. Good examples of the power of enriching communities are 
the Cavendish Laboratory at Cambridge University with its 30+ Nobel laureates, the 
Linux operating system that is developed by a virtual community of users in enriching 
interaction, and a jazz session that is based on improvisation. 

Pekka Himanen, a well-known Finnish philosopher, talks about enriching cultures 
(Himanen, 2007). He says that a good conversation is an enriching interaction where 
people help each other to be their best and rise further than anyone could go alone. It is 
to understand that helping others to succeed is the best strategy to be successful yourself. 
In order to create enriching cultures in schools, there needs to be trust and mutual 
responsibility in the community of educators and students. Trust is the foundation on 
which a community builds its being and behavior. When there is no trust in the 
community, people “fall through that foundation to the stage of fear” and use their energy 
on coping with insecurity. An enriching community emerges on trust, security, and 
togetherness - all core values of education. 

An enriching community and interaction that drives it are conditions for risk taking, 
ingenuity, and creativity. It requires that each member in the community is valued, i.e., 
that each experiences concretely that he or she is important and needed in the community 
which enhances the feeling of belonging. Schools and classrooms can be or become an 
enriching community if teachers and principals want them to be. A culture of 
competition is often harmful in efforts to build enriching community. Collaboration and 
constructivism are positive ways to enrich communities. Therefore, school improvement 
should enable teachers and students to learn and create conditions for enriching 
interaction. Having time for these important conditions is a necessity. Figure 16.1 
presents ways to build enriching communities. 
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Figure 16.1. Building Enriching Communities 

Creativity in school is only possible if people, both students and teachers, are encouraged 
to take risks. This requires safe and secure communities where mistakes and effort are 
rewarded as much as achievement. Creativity brings joy and increases meaningfulness in 
school when people can fulfill their personal wishes and create new ideas and experiences 
for themselves and others. 

A creative culture of learning is the ultimate goal of school improvement. It is driven by 
a personal goal, vision, or dream. It is stimulated by positive emotional energy that, in 
turn, is a source of renewal and change. A dream is a counterforce of fear (or nightmare). 
That is why in school improvement we need not only rational goals and measurable 
targets, but also a compelling vision and inspiring dreams. 

Policies for Sustainable School Improvement 

Policies that support school improvement should focus on developing three elements: 
creative and competent people, productive structures (i.e., schools and learning 
environments), and the creative cultures of learning. Education leaders should make sure 
that schools have enough creative and competent people. In other words, it is important 
to have a critical mass of ‘enrichers’ in school, not only enough bright and active people. 
Enrichers are those who know how to create enriching interaction and who can work 
with leaders to build enriching communities in schools. Second, school improvement 
should support and create structures that enable people to communicate, meet, and 
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exchange ideas. Communication can vary from electronic interaction to informal teams 
and formal conferences. Productive structures are necessary elements of school 
improvement. It is important that teachers, students, and school leaders have 
opportunities for informal interaction and communication outside of formal professional 
contexts. Third, school improvement should promote building trust in schools and 
thereby avoid the spread of insecurity and fear in the school. Fear, frustration, and 
hopelessness lead to development of different coping strategies that suck energy from 
individuals. For example, it is a common phenomenon in many organizations that 
individuals play several coping games in order to survive in insecure situations. A person 
may play a game of not making a fool of himself in front of others; this is a typical 
coping strategy. It takes a lot of energy to keep that strategy up, especially in problem- 
solving or creative situations. 

Four common aspects that characterize successful schools and communities follow. 

1. School has a vision or preferably a dream. This dream needs to be emotionally 
energizing and shared by the vast majority. Ironically, most of the corporate visions 
are boring and lack the necessary emotional power that makes a dream inspirational. 
In school improvement this means that education leaders need to work toward a 
shared vision for all schools that then turns it into a dream at the level of schools. 

Only then will teachers and students engage in and invest energy to achieve their 
common goals. 

2. There is creativity in school. Creativity requires freedom. Taking risks and doing old 
things in a new way are important starting points to creativity. The school principal 
doesn’t need to do everything and be everywhere. Freedom needs to be earned 
through making things happen by being flexible and free of unnecessary control; 
freedom is not a self-evident right of teachers or students. And, as noted earlier, trust 
in schools and classrooms is a necessary condition or foundation upon which 
creativity can be built. 

3. School is an enriching community. There is no enriching interaction in classrooms 
unless there is enriching interaction among teachers. Teachers need to work together 
and feel that they are part of an enriching community where collectively they are 
more than themselves. The teachers’ staffroom has to be an encouraging and 
inspiring place where everyone can be who they are. Individual energy is used for 
improvement, not in coping strategies or survival games. Leadership must guarantee 
that there is a critical mass of enrichers in schools. 

4. School has a culture of trust. Trust is the foundation on which everything in a 
creative community rests. School needs to be a safe and secure place where 
differences are seen as a source of energy and renewal rather than reasons for conflict 
and distrust. Trust can never be mandated or regulated; it has to be built by acts and 
behavior. One can only be trusted by being trustworthy. 
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Figure 16.2. Community-Energy Tensions in School Improvement 



School improvement can lead the way to build ideal cultures of learning in schools by 
focusing on enhancing the use of emotional energy and building enriching communities 
in schools. Figure 16.2 describes the tension between the intentional state of emotional 
energy and enriching community on one side, and emotional vacuum and impoverishing 
community on the other side. 

When I asked Pekka Himanen what the essence of enriching interaction is, he told me 
that it is when one can adopt the tone of the other. A test of this is to see if you can sense 
what the other person has always longed for. School improvement policies - if the 
community-energy tension is taken seriously - should provide more room for genuine 
dialogue and collaboration in schools. Inability to do so may have very negative long- 
term effects. Apathy, low self-esteem, and fear are some of the features of a culture 
where dialogue and collaboration have vanished and are replaced by monologue and 
isolation. 
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The Dream 



I have a dream is a universal slogan. It has meaning in certain times and situations in our 
history. Imagine Martin Luther King Jr. standing on the steps of the Lincoln Memorial 
saying, “I have a dream that our economy will grow at the rate of five percent each year”. 
He didn’t say that. His dream was, “I have a dream that my four little children will one 
day live in a nation where they will not be judged by the color of their skin but by the 
content of their character.” It was the inspiration and hope that kept tens of thousands of 
people listening to these words that are today more relevant and inspiring than ever 
before. 

I have a dream, too. My dream is not that “one day every school will make adequate 
yearly progress as measured by student achievement tests” or anything similar. I have a 
dream that in future our schools, their students, and teachers can live in the culture of 
trust. I have a dream that one day our teachers can work in a spirit of responsibility 
rather than accountability. I have a dream that soon all our children will learn in truly 
enriching communities. School improvement has a key role to play in reaching out to 
this dream. At her murdered husband’s funeral Yoko Ono said, “If you have a dream, it 
is just a dream. But if you share it with other people, it will come true.” That is the 
purpose of my presentation to you. One day soon, I dream, we will have a genuinely 
caring and creative school where passion to learn and responsibility to help exist for all 
our children. 
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Chapter 17 

Educational Innovation and Complexity 2.0: Some Implications for AISI 



Brent Davis and Dennis Sumara 
University of British Columbia 



This chapter consists of three parts. In the first, we describe the transdisciplinary movement 
of complexity research, focusing on two key moments that we call Complexity 1.0 and 
Complexity 2.0. In the process, we ask two entangled questions: In what ways and to what 
extent might AISI be understood as a complex system? In the final part, we invert the focus, 
exploring how complexity research - and, in particular, the emergent field of inquiry of 
network theory - might be used to inform and transform the structures of and possibilities for 
AISI. 



Complexity Research 1.0 

Some Attempts at a Definition 

As researchers, we came across complexity theory in the 1980s. We immediately felt 
that complexity had an important contribution to make to educational theory, research, 
and practice. In this section, we elaborate this hunch, using AISI as both illustrative 
example and object of analysis. 

Complexity theory (more recently, complexity science, complexity thinking, or simply 
complexity) is the title chosen by a group of researchers from many different disciplines 
who first came together in the 1970s around the realization that the assumptions, 
methods, and metaphors of classical analytic science were inadequate for the study of a 
particular sort of phenomenon. Specifically, it was noted that some ‘things’ - including 
hearts, ice ages, and stock markets - share some surprising qualities. 

For example, as illustrated in Figure 17.1, when the dynamics of these sorts of phenomena 
are considered in appropriate time scales, they demonstrate some curious similarities. Their 
profiles are not smooth and steady trajectories, but jagged traces with occasional dramatic 
transitions. The principal reason for such similarities is perhaps unsurprising: complex 
phenomena are not islands unto themselves. They are, rather, coupled to many other 
phenomena. The profiles that we see are indications of ongoing adaptations to contexts 
that are similarly dynamic and active. 
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Figure 17.1. Three Phenomena that, When Viewed in Appropriate Timescales, Show Complex 
Profiles 

The manner of comparison might be described as comparative dynamics (Stanley, 2006), 
which might in turn be contrasted with the much more familiar comparative statistics. 

Derived from Latin status, “make (or be) firm,” statistics is a domain developed 
principally to compare snapshots of forms or phenomena that are similar, such as the 
heights of different adult males or the activities on different stock markets. 

To extend the photography analogy, whereas comparative statistics deals with snapshots, 
comparative dynamics is more about comparing videos, concerned with real time events, 
and ongoing similarities and differences among active systems - a sense that is implicit in 
the phrase “comparative dynamics.” Derived from the Greek dynamis, “to have power or 
energy,” dynamics is about continuous change and adaptation. Comparative dynamics - 
or complexity research - is thus organized around the behavioral and evolutionary 
similarities among very different phenomena. 

The importance of this distinction comes into better view when one contrasts the sorts of 
distributions associated with comparative statistics and comparative dynamics. The 
distributions associated with the former are a family of forms that are similar to the 
familiar normal curve, whereas phenomena associated with the latter are more typically 
associated with power law distributions (see Figure 17.2). 





Figure 17.2. The Normal Distribution of Comparative Statistics Contrasted with the 
Power Distribution of Comparative Dynamics 



AISI Colloquium 



157 



Large Scale Improvement 



Normal distributions require a ‘normal’ - an arithmetic mean around which data points 
are assumed to cluster in a particular, predictable manner. In striking contrast, with 
phenomena that obey a power law distribution - such as earthquakes, wealth, wars, 
learning systems, and learning events - the notion of arithmetic average makes no sense 
at all. One can certainly determine an average wealth of a person on earth, for example, 
but when the few dollars of some billions are combined with the billions of some few, the 
result is a meaningless and worrisomely misleading datum. 



In investigations of how and why power law distributions arise, complexity researchers 
have proposed an important distinction between complicated and complex phenomena. 
Illustrated and summarized in Figure 17.3, complicated phenomena are ones that can be 
described and confidently predicted using the laws of physics. Complex phenomena, 
however, are not quite so predictable or controllable. Better characterized in terms of 
recursive cycles than linear flowcharts, complex systems can differentiate and self- 
amplify and/or self-suppress in ways that inevitably give rise to power law distributions. 



complicated vs complex 




Mechanical (Newton) 
Machine metaphors 
Linear movement 
Input/output flowcharts 
Efficiency-seeking 
Progress-minded 
Reducible 



Adaptive (Darwin) 
Ecosystem metaphors 
Exponential change 
Feedback loops 
Sufficiency-oriented 
Growth-minded 
Non-compressible 



Figure 17.3. Some Differences Between Complicated and Complex Phenomena 



Underlying these capacities to adapt and self-amplify are qualities that are common to a 

range of dynamic phenomena. For example, all complex systems are: 

• Self-organizing or emergent (they are ‘bottom-up’; coherent unities can arise without 
a centralized controller) 

• Self-determining (the way that a complex system responds is determined by the 
system itself, not the perturbation on the system) 

• Far-from- equilibrium (complex forms have somewhat erratic profiles; predictable or 
stable profiles indicate ill health or imminent death) 
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• Outfluence is disproportionate with influence (popularized as ‘the butterfly effect,’ 
the extent of the consequences of minor - and major - events are unpredictable, 
rendering linear constructs such as “value for money” highly troublesome) 

• Nested (complex forms unfold from and are enfolded in one another) 

• Scale independent (whether magnified or reduced, complex forms maintain a constant 
bumpiness of detail) 

This is only a partial list. Among many other important qualities that have been 
identified are recursive elaboration, porous boundaries, and networked structures (see 
Davis & Sumara, 1997; 2006). 

Briefly, then, the word complexity refers to an emergent recognition that some phenomena 
cannot be described or understood in terms of mechanical metaphors, cause-effect logics, 
or linear images - a statement that should not be read as a rejection of the classical 
scientific method. Complexity does not discard an analytic attitude. Rather, it highlights 
the limitations of and seeks to elaborate that attitude toward research that frames 
phenomena in terms of predictable sums of their parts. 

And so the point is not that modern science is in any way misguided; it is that the tools of 
reductive analysis can only take us so far in studies of, for example, large-scale economies, 
ecosystems, and brains. These sorts of phenomena are complex. By way of preliminary 
(but inadequate) description, a complex unity is one that arises in the interactions of other 
systems as these give rise to actions and traits that are not possible independently. In this 
sense, economies cannot be reduced to consumers, ecosystems are more than collections of 
species, and brains transcend neurons. Such phenomena are better characterized in terms 
of adequacy rather than efficiency, exponential growth rather than linear progress, and 
dynamic possibility rather than static potential. They call for holistic, contingent, and 
exploratory approaches to inquiry. 

In fact, in a strong sense, a complex phenomenon ‘dictates’ how it must be studied. In 
other words, there are no generalized, universal methods within complexity. This point 
helps to explain why complexity resists precise and concise definition. Complexity 
research is conceived more in terms of what one investigates than how one investigates. 
Hence, biologists tend to talk about complexity in terms of living systems, physicists in 
terms of nonlinear dynamics, and economists in terms of micro- and macro-economies. 
Perhaps not surprisingly, then, as educators and educational researchers we find ourselves 
gravitating toward a particular interpretation: complex systems are systems that learn. 



Key Moments in the Emergence of Complexity Thinking 

Since cohering as a movement within academia, complexity research has moved through 
several distinct emphases, discernible according to emphases, metaphors, and research 
strategies. In the first phase, coinciding with the emergence of the movement, the focus 
was on gathering rich descriptions of varied instances of complexity. Jane Jacobs’ 

(1961) accounts of the death and life of cities and Deborah Gordon’s (1999) life histories 
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of anthills are two commonly mentioned exemplars of this stage, during which there was 
a powerful intuition that these phenomena were related but no clear consensus on how the 
relationships might be articulated. 

In the next stage, during the 1980s and 1990s, the initial emphasis on description was 
elaborated to include efforts at explication. Researchers began to identify and characterize 
common underlying dynamics and, enabled by rapid and inexpensive modeling technologies, 
began to develop common principles of interpretation and rules of behavior for complex 
phenomena. 

So successful was the explicative stage that by the mid-1990s researchers had begun to 
investigate the pragmatics of new insights, exploring formerly intractable problems such as 
the restoration of vanquished ecosystems, restructuring of multinational corporations to 
enable these massive organizations better able to adapt to rapidly changing circumstances, 
and so on. It was at this point that we began to notice a modest, but serious uptake of 
complexity ideas among a handful of educational researchers, we believe because 
complexity research was perceived by at least a few to be aligned with the deliberate 
project of education. 

Among the many insights generated during this stage was an ever-growing list of 
‘conditions’ necessary for the emergence and maintenance of complex coactivity. In our 
work, we have found it useful to select a subset of these conditions and present them as 
simultaneous, co-amplifying pairs - that is, conditions that come together in complex 
harmony rather than mechanical balance. We have found the following to be particularly 
useful to our work as researchers and teachers: 

• Agents are specialists and agents are generalists. Human social organizations 
require specialists, at the same time as all participants in a collective must be able to 
generalize and adapt their capacities. Among humans the best specialists are 
typically the most adaptable generalists, and the best generalists are typically those 
who have developed a deep expertise in some domain. 

• Internal redundancy and internal diversity. Redundancy - the commonalities in 
worldviews, responsibilities, expectations, insights, etc. among agents in a system - is 
the source of a system’s robustness. It is what enables agents to work together and to 
fill in for one another. Diversity - briefly, the range of responses that a system can 
drawn on in new situations - is the source of a system’s intelligence. 

• Systemic flexibility and systemic rules. For a system to learn, it must be flexible and 
adaptable. In particular, it must not be governed by prescriptive rules that dictate 
what must be done. However, there clearly must be rules. In instances of complexity, 
these constraints are proscriptive, usually expressed in terms of what must not be 
done. As exemplified by the Ten Commandments or the rules of a soccer game, 
systemic rules that are articulated in terms of the boundaries of play are constraints 
that can actually enable. 
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• Independence among agents and interdependence among agents (‘Self versus 

society’ is a classic tension in educational thought. Complexity research challenges 
this simplistic dichotomy by highlighting that, in human systems, the settings in 
which agents typically have the most autonomy are also the ones in which they rely 
the most on one another. That is, independence and interdependence are not at odds 
in a complex system; they are mutually amplifying. 

Based on our analysis of AISI materials, we feel that the above conditions are well in place 
in most of the levels of organization (i.e., school, district, and provincial levels) that are 
represented in reports and policy statements. However, the extent of their representation 
seems to have varied from one cycle to the next. In Cycle 1, for example, there seemed to 
be a much stronger expectation of general improvement through specialization in AISI- 
funded projects, embodying recognition that there are typically generalized benefits from 
specialized expertise. By Cycle 3, with a dramatic reduction in the foci of research 
projects, the potency of the dynamic specialization-generalization coupling seems to have 
been diminished. Similarly, whereas there appeared to be considerable diversity in Cycle 1 
of AISI that range of focal interests seems to have decreased markedly by Cycle 3 - 
prompting us to wonder if that is a dimension that AISI might want to nurture in the next 
cycle. 

It is a bit more difficult to discern the relative roles of flexibility and rules within AISI, in 
large part because of the very different ways that AISI has been taken up in different 
districts. (We hasten to add the possibility of such variety is likely a very strong feature, 
in complexity terms.) On this matter, then, we are compelled to limit our commentary to 
key questions for ongoing review and reflection: What are the musts? Do they constrain 
possibilities by being too prescriptive? When are these too vague or too implicit? How 
are the rules adapted on the fly? 

Finally, on the matter of independence and interdependence, we note that although 
individual projects are given considerable creative autonomy, the required links between 
and among projects aren’t obvious - nor, for that matter, does it appear that they are 
necessary to the success of any given project. On this count, we wonder if AISI might be 
missing some rich opportunities to serve as a uniting force - a knowledge-producing 
collective, in effect - because there has been limited attention given to the interdependent 
aspects of varied research initiatives. We develop this point in Parts 2 and 3. 



Complexity Research 2.0 

In their review of the emergence of technologies that enable rapid and large-scale social 
networking, Lankshear and Knobel (2006) develop a distinction between what they call 
Mind-set 1.0 and Mind-set 2.0. 

Two apt descriptors of Mind-set 1.0 are collective and representative. This mind-set is 
collective in the sense that, through such technologies as the internet and mobile 
communication devices, huge numbers of people have been able to come together to 
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share in real time what they know about a stunning array of topics. It is considered 
representative because this sharing has principally been in the form of distribution of 
established insights, as opposed to the active and engaged collaborative elaboration of 
what is known. 

The sort of active, collaborative elaboration is the hallmark of Mind-set 2.0, which might 
be characterized in terms of connectivity and participation. Exemplars of this mode of 
engagement are Wikipedia and Facebook - sites in which participants not only collect, 
they connect around topics of shared interest. Jenkins and his colleagues (2007) refer to 
such engagements as “participatory learning cultures,” which have the following 
qualities: 

• There are opportunities for expression and engagement. 

• There is support for creating and sharing creations. 

• There is some type of teaching so the most experienced can mentor new members. 

• Members believe their contributions matter. 

• Members feel social connection with one another. 

• Group memory is a collective responsibility (but one that often becomes a specialized 
role). 

There are some striking similarities between the entries on this list and the set of 
entwined conditions of complex emergence mentioned in the previous section - which 
should not be surprising. Jenkins’ list is a re-articulation of the conditions of complex co- 
activity as narrowly applied to human social systems. 

Such shifts are being realized across domains in academia, as the myth of the “great 
mind” that propels society through sudden invention is being replaced by a narrative of a 
“mass mind” that is constantly innovative. The point is captured by Harvard biologist 
E.O. Wilson (2002) in an interview in which he explains why he believes that complexity 
science is where we are going in the 21 st century: 

Today it is harder for a scientist to single-handedly make new discoveries. Rather, the 
future of science lies in a collaborative synthesis of existing data to understand 
interactions within complex systems at all levels beyond the level of the single cell. 

This transition is represented within complexity research through the recent emergence of 
network theory. Indeed, as illustrated in Figure 17.4, the main visual metaphor of the 
movement seems to have shifted from co-implicated eddies to a de-centralized network. 

This point bears some elaboration. In Figure 17.5, we present three identical sets of dots 
that have been linked in three distinct ways to create three very different types of 
network. The first, in which a node in the middle is selected as the hub of the network, 
depicts a centralized network, which is a structure that has the advantage of very effective 
distribution of resources and/or information, but the disadvantages of not being very 
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robust or adaptable since these qualities are determined by the central hub alone. If the hub 
collapses or fails to adapt, the whole system is compromised. Despite these potential 
problems, however, it remains the network of choice when the aim is efficient distribution 
of information or resources (e.g., over an intercom, in a lecture, or through a provincially 
funded initiative). 



Figure 17.4. A Shift in Visual Metaphor from Co-implicated Cycles of Complexity 1.0 (with 
a Focus on Collectivity) to Networks of Complexity 2.0 (with a Focus on Connectivity) 




Figure 17.5. Three Types of Network - Centralized, Distributed, and Decentralized 

A distributed network, in which each node is connected to its nearest neighbors, solves the 
problem of robustness. Like the roadway grid in a city, even when many nodes or links are 
‘knocked out’ in such a network, it can continue to function. That extreme robustness, 
however, comes at the expense of quick communication and high adaptability. Such 
systems tend to suffer from lock-in, since each node is relatively isolated from the grander 
network. 

A decentralized network is a sort of hybrid of centralized and distributed networks in 
which there are many ‘centres’ (or ‘hubs’), each of which is well-connected to some 
nearby neighbors. Extensive research and modeling has demonstrated this to be the 
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essential structure of all complex phenomena - manifest in, for example, the structures of 
airline routes, memory, languages, ecosystems, the spread of disease, social systems, and 
the internet. Harmonizing efficient communication, robust structure, and adaptability, the 
decentralized network is the “fingerprint” of intelligent learning systems. (Note that the 
image presented is highly simplified. More nuanced depictions of these structures can be 
viewed at www.visualcomplexity.com .) 

It is important to stress that decentralized networks can only exist in some sort of 
dynamic tension and are constantly under threat of ‘decaying’ into less adaptable, less 
intelligent centralized or distributed networks. For instance, as often witnessed with the 
introduction of high-stakes examinations, decreases in classroom time, or introductions of 
more rigidly structured curriculum, a sudden stress on an education system can ‘force’ 
dramatic shift toward teacher-centered methods (i.e., centralized networks). Conversely, 
the tendency toward more individual-centered structures is often associated with the 
removal of standardized evaluations, excessive resourcing, and increased choice. Simply 
put, there are constant needs for productive stress, adequate support, and enabling 
constraints to maintain decentralized structures. 

To our analysis, all three of the networks illustrated in Figure 17.5 are manifest in AISI. 
For example, the funding structures and the growing database of results appear to us to be 
centralized networks. By contrast, the “on the ground” research projects seem to be 
undertaken in distributed networks, in which there appears to be little need for schools 
and/or jurisdictions to network with others in projects that demand more nuanced, 
broader-based collaborations. A clear and well-functioning decentralized network is 
evident in the domain of administrative organization, with teachers clustering into school 
collectives, schools into a district collective, and districts into the provincial network. 

However, and somewhat troubling to our analysis, AISI does not seem to have coalesced 
into a decentralized network as a knowledge-producing collective. Rather, in our view, 
the prevailing structure seems to be a fragmented network (see Figure 17.6) in which 
critical links among localized collectives have not been established. 




Figure 17.6. A Decentralized Network Contrasted with a Fragmented Network 
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Deliberate About More Complex Possibilities 

In this final section we move toward some pragmatic recommendations for AISI. The 
first word in the title of this section, deliberate, was chosen for its dual meaning. On one 
side, as an adjective, deliberate describes action with careful consideration that is done on 
purpose or intentionally. It is typically slow and consciously unhurried. On another side, 
the verb deliberate is about collaborative consultation with others in a reflective and 
recursive process of selecting what might be done next. Our guiding questions here are: 
Where might AISI be deliberate? And How might AISI deliberate? 

On this matter, we hasten to qualify our status as ‘experts’ in this realm. It is not a title 
we would choose ourselves, except in the senses the word shares with its cognates 
experience and experiment. Derived from a root that has to do with trying and learning 
through testing, we note that our expertise in this domain is a tentative sort. Indeed, it is 
really the only sort that complexity research allows for. There can be no certainty in 
efforts to trigger the emergence of complex collectivity/connectivity. 

Our tactic in this section is to draw on and further develop the notion of the “consciousness 
of the collective” (Davis & Sumara, 2006). This metaphor is anchored to a century of 
intense research into the nature of individual consciousness, through which it has been 
demonstrated that consciousness is very much unlike the way it is typically conceived. It is 
not all-perceiving, in charge, forward-looking. On the contrary, consciousness is more 
commentator than a controller - more like a sports broadcaster reporting what just happened 
on the soccer field than a coach or a player who can directly affect what might happen next. 
It turns out that consciousness is always running a little behind the action and that it is 
capable of juggling only a tiny, tiny fraction of the millions of perceptual possibilities that 
impinge on the senses each instant. But none of this is to say that consciousness is 
unimportant. On the contrary, in orienting attentions of the grander system, consciousness 
plays a vital role. What we are, what we know, and what we do are all utterly dependent on 
consciousness. However, none is determined by consciousness. 

The relevance of this point might be made clearer through a few brief examples. Consider, 
for instance, the role of the news media in our society. They act as a sort of consciousness 
of the collective by selecting, interpreting, and re-presenting a narrow slice of what is going 
on in the world. In the process of orienting attentions to these selections, interpretations, 
and re-presentations, they play a powerful role in affecting what we imagine ourselves to 
know, do, and be. That is, our collective possibilities are utterly dependent on the 
amplifications provided by our media, although they are in no way determined by those 
media. 

In a similar sense, we would argue, the teacher acts as a sort of consciousness of the 
collective in the classroom. That is, the teacher is constantly selecting from, interpreting, 
and re-presenting possibilities that arise in the space of collective activity. Similarly, the 
principal acts as the consciousness of the collective of the school, the superintendent as 
the consciousness of the collective of the district, and so on. 
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And it is here that we come to perhaps our most urgent question around AISI: Who and 
where are the consciousnesses of the collective in the AISI knowledge-producing 
collective? As noted above, it appears that such nested levels of consciousness are in 
place and functioning well in the domain of educational administration. But it is not so 
clear that they are present and working in the domains of collective knowledge- 
production and transfer. Illustrating our point with Figure 17.7, it is our feeling that, 
whereas individual schools and districts are doing powerful and important work, the vital 
‘connective tissue’ is missing at the critical membranes that separate one school from 
another and one district from another. The absence of these critical links serves to limit 
the potential of AISI by fragmenting the grander network (or, perhaps more accurately, 
by not providing the means to pull together a decentralized knowledge-producing 
network). 
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Figure 17.7. Combining the Visual Metaphors Presented in Figure 17.4, a Graphic 
Intended to Highlight the Critical ‘Missing Links’ at the Membranes of Functioning 
Knowledge-producing Collectives within AISI 



This figure also prompts our attentions to several other issues, including some of the 
other overlapping and interlacing systems that are not included in the image. For 
example, drawn as it is, the image seems to suggest that formal educational structures 
exist independently of parents, communities, and other social institutions. We find an 
even more drastic ‘compression’ around the oft-mentioned aim of AISI to “improve 
student learning,” which seems to compel attentions to individuals and to force an 
uncomfortable emphasis on a narrow category of performance-based and measurable 
evidence. 

Complexity research, we believe, should prompt quite a different frame, which might be 
described in terms of “improving learning” rather than merely “improving student 
learning.” This shift is not a subtle one. As we attempt to illustrate in Figure 17.8, the 
three nested domains of schools, districts, and the province might be construed as the 
edusphere, which itself is enfolded in and unfolds from other levels of complex activity. 
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Figure 17.8. Some of the Nested Systems, Domains of Inquiry, and Major Discourses in 
Contemporary Educational Research, Illustrating the Transphenomenal, Transdisciplinary, 
and Interdiscursive Nature of the Enterprise 



In complexity terms, all of these levels are complex learners - emergent, self-determining, 
nested, disequilibriated unities. Further, various aspects of and projects within AISI address 
these learners in important ways. 

To make sense of this assertion, we must be careful to situate each of these phenomena in 
its appropriate time frame. Individual learners (i.e., egowebs) learn in minutes, hours, 
days, weeks, and years. Different aspects of the eduweb learn in months, years, decades, 
and centuries. The ethnoweb and ecoweb (ignoring the ever-present possibility of sudden 
and massive cascading failure) typically learn in correspondingly longer time frames. 

This realization of temporal variability among learning systems is of critical importance 
to issues of methodology and evidence when researching complex phenomena. As we 
intend to suggest by the “Ways to Study Learning” column in Figure 17.8, complexity 
research presses attentions beyond narrow conceptions of “evidence” that are tethered to 
the psychologistic emphases implicit in efforts to “improve student learning.” Rather, 
schools and districts - and families, classroom collectives, and communities - are also 
learning, calling for at least some sociological research. To study phenomena in the 
eduweb, for example, prospective researchers can draw on rich methodological histories, 
utilizing ethnographic methods to develop rich and nuanced accounts of the learnings of 
schools and districts, or hermeneutic methods to interrogate and interrupt systems of 
belief and interpretation that frame schooling cultures. The same sorts of argument apply 
to anthropological and ecological studies. Unfortunately, to our analysis, these layers of 
research are virtually unrepresented in the current AISI literature. (Notably, an action 
research model can readily accommodate these insights; see Sumara & Davis, 1997). 
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At the same time, as flagged by the “Discourses on Learning” column in Figure 17.8, 
complexity research offers a means to bring the many and varied frames in education into 
conversation with one another. The critical question for complexivists is not around the 
differences in theories and perspectives, but the level of learning that specific theories were 
developed to understand. For example, it makes no sense within a complexity frame to ask 
which of constructivism, social constructionism, or critical theory is most appropriate to 
education - since these are focused on distinct phenomena (i.e., respectively, in the terms 
presented in Figure 17.8, the egoweb, the eduweb, and the ethnoweb). 

We’ll use this point as a transition into our more focused recommendations for AISI, all 
of which are in some manner linked to the notions of transphenomenality (i.e., the nested, 
scale-independent natures of complex phenomena), temporal variability (i.e., the 
realization that different systems learn in particular, but sometimes overlapping 
timescales), transdisciplinarity (i.e., the necessity of bringing together research expertise 
across a variety of domains), and interdiscursivity (i.e., bringing diverse discourses into 
conversation around a matter of shared interest). But before offering our specific 
recommendations, we reiterate a vital qualification, namely that one cannot force 
complexity into existence. More appropriate verbs include occasion, foster, and nurture 
- pointing toward a need for tentative and attentive action, not legislated or scripted 
moves. This realization that one cannot pre-determine or prescribe what a self- 
determining system will learn/do (i.e., how agents/systems will respond to perturbations) 
is at the same time a source of potential frustration and a site of possible fecundity. 

Leaving behind the hope of prescribing outcomes, we are left with considerations of 
proscribing the boundaries of various engagements - that is, considering the conditions 
that might be put into place and the structures that have been or that might be created. 

What sorts of enabling constraints, productive stresses, and adequate-but-not-excessive 
resourcing might set the stage for the emergence of a decentralized learning system? 

We don’t mean to be dramatic with this question. In our assessment, AISI might well be 
positioned to move from an interesting professional development initiative that is 
characterized by local but disconnected projects into a knowledge-producing network on a 
scale that has not been seen in formal education. Elaborating the advice, mentioned above, 
to identify places in the system where there are few or inadequate links among agents and 
levels in the system, we would urge consideration of the following conceptual elements: 

• Identify and resource links within, across, and among ‘agents’ at the various levels of 
organization within AISI. 

• Support specialization and differentiation of roles within networks (e.g., 
commentators, interpreters, effecters, etc.) and nurture ‘focal practices’ (focused, often 
idiosyncratic interests) in the network. 

• Infuse research expertise into the system from across relevant disciplines, and enlarge 
current understandings of evidence, data, and proof by welcoming diverse research 
strategies, methods, and interpretative frames. 
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In more pragmatic, action-oriented terms, we recommend the following actions: 

1. Request additional targeted funding to hire research specialists to act as 
consciousnesses of the AISI collective - responsible for monitoring, interpreting, and 
re-presenting how the entire system is (or is not) learning. 

2. Consider targeted funding to support specialized connectors who will liaise and 
facilitate knowledge transfer among districts. 

3. Build more diversified research teams that include specialists. 

4. Be more explicit and critical about theoretical commitments in planning and reporting 
of research (in particular, be attentive to the current tendency to mix complicated and 
complex sensibilities). 

5. Initiate and support robust research into phenomena within the eduweb (see Figure 
17.8), taking care (a) to utilize methods appropriate to those phenomena and (b) to 
avoid the conflation of efforts to affect the eduweb and data based on individual 
performances (i.e., egowebs). 

Educational action and educational research are always and already sites of learning, and 
it is thus that we believe they might be productively wedded to the complexivist interest 
in the study of learning systems. Many complexivists within education have been 
working to do precisely this: to tease out how particular complex learning systems cohere 
and unfold. How do classroom collectives emerge? What is the structure of a body of 
knowledge? How are individual knowers implicated in collective knowledge? When 
considering educational research from a complexity research perspective, it is important 
to consider not just the original concerns and the desired ends, but the immediate 
conditions for collective action. There is never any guarantee that complex collectivity 
will emerge, but failing to attend to the necessary conditions for complex co-action will 
almost certainly ensure that it will not arise. 
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